2024-05-27T12:21:15.179 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T12:21:15.179 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886 --verbose --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} --name rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886/orig.config.yaml 2024-05-27T12:21:15.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T12:21:15.332 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7727886' kernel: client: branch: testing flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 81797d879079e1a392b37a9c23a946594f658c50 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 81797d879079e1a392b37a9c23a946594f658c50 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-clone-stats sha1: 762bfab4bbf713ca5c79093d382422db341b04da owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 791 sha1: 81797d879079e1a392b37a9c23a946594f658c50 sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-clone-stats suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_762bfab4bbf713ca5c79093d382422db341b04da/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 762bfab4bbf713ca5c79093d382422db341b04da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFxzH66mEcX5t06aaIW/gkpvBw/ovdyjzzw6ljDcUZZat7XFtuABWNAwi6Am6NP/ngDYsLl5eKxgU4NwE4Q3qhc= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNOvNGkFAta5F0vke0nI+YnFLVGMO2EgTML2+cceEjfLRz55tefDqZ7r+U7BEelA7dCEl6/C8DpNkf8aPYol0U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_12:15:13 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T12:21:15.332 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_762bfab4bbf713ca5c79093d382422db341b04da/qa; will attempt to use it 2024-05-27T12:21:15.333 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_762bfab4bbf713ca5c79093d382422db341b04da/qa/tasks 2024-05-27T12:21:15.333 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T12:21:15.335 INFO:teuthology.task.internal:Checking packages... 2024-05-27T12:21:15.365 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '81797d879079e1a392b37a9c23a946594f658c50' 2024-05-27T12:21:15.365 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T12:21:15.365 INFO:teuthology.packaging:ref: None 2024-05-27T12:21:15.365 INFO:teuthology.packaging:tag: None 2024-05-27T12:21:15.365 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-05-27T12:21:15.365 INFO:teuthology.packaging:sha1: 81797d879079e1a392b37a9c23a946594f658c50 2024-05-27T12:21:15.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-clone-stats 2024-05-27T12:21:15.506 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.g81797d87 2024-05-27T12:21:15.507 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T12:21:15.515 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T12:21:15.515 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T12:21:15.540 INFO:teuthology.task.internal:Saving configuration 2024-05-27T12:21:15.558 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T12:21:15.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T12:21:15.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 12:16:20.931360', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFxzH66mEcX5t06aaIW/gkpvBw/ovdyjzzw6ljDcUZZat7XFtuABWNAwi6Am6NP/ngDYsLl5eKxgU4NwE4Q3qhc='} 2024-05-27T12:21:15.646 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 12:16:20.929711', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNOvNGkFAta5F0vke0nI+YnFLVGMO2EgTML2+cceEjfLRz55tefDqZ7r+U7BEelA7dCEl6/C8DpNkf8aPYol0U='} 2024-05-27T12:21:15.646 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T12:21:15.654 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-27T12:21:15.654 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-27T12:21:15.654 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T12:21:15.713 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fef316e7be0>, signals=[15]) 2024-05-27T12:21:15.713 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T12:21:15.721 INFO:teuthology.task.internal:Opening connections... 2024-05-27T12:21:15.721 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:21:15.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:21:15.813 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:21:15.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:21:15.891 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T12:21:15.899 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-05-27T12:21:15.916 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-05-27T12:21:15.917 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-05-27T12:21:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-05-27T12:21:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-05-27T12:21:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-05-27T12:21:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-05-27T12:21:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T12:21:15.975 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T12:21:15.976 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-05-27T12:21:15.993 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-05-27T12:21:16.011 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-05-27T12:21:16.011 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-05-27T12:21:16.068 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T12:21:16.069 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T12:21:16.069 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-05-27T12:21:16.093 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T12:21:16.104 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T12:21:16.113 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T12:21:16.113 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-05-27T12:21:16.116 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-05-27T12:21:16.132 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T12:21:16.140 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T12:21:16.140 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-05-27T12:21:16.172 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-05-27T12:21:16.368 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T12:21:16.424 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T12:21:16.437 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T12:21:16.438 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-27T12:21:16.438 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-27T12:21:16.438 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T12:21:16.439 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T12:21:16.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:21:16.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-27T12:21:16.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:21:16.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-27T12:21:16.661 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:16.661 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T12:21:16.662 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-05-27T12:21:16.675 DEBUG:teuthology.task.kernel:sha1 for client.1 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:16.675 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T12:21:16.675 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-05-27T12:21:16.679 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-05-27T12:21:16.679 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:16.679 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T12:21:16.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum upgrade -y linux-firmware 2024-05-27T12:21:16.692 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-437.el9.x86_64 2024-05-27T12:21:16.692 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:16.693 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T12:21:16.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2024-05-27T12:21:18.950 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-27T12:21:20.690 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-27T12:21:22.265 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.1 MB 00:03 2024-05-27T12:21:24.016 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-27T12:21:25.299 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-05-27T12:21:27.450 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 17 kB 00:00 2024-05-27T12:21:28.491 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-05-27T12:21:30.067 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 17 kB 00:00 2024-05-27T12:21:31.389 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-27T12:21:36.536 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-27T12:21:38.533 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-05-27T12:21:38.534 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-05-27T12:21:38.534 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-05-27T12:21:38.654 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-27T12:21:38.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:38.879 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T12:21:38.879 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T12:21:38.882 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-05-27T12:21:47.552 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-27T12:21:52.641 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-27T12:21:54.645 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-05-27T12:21:54.646 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-05-27T12:21:54.646 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-05-27T12:21:54.750 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.1... 2024-05-27T12:21:54.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T12:21:54.972 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T12:21:54.972 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T12:21:54.975 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-05-27T12:21:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27 12:21:55 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T12:21:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-05-27 12:21:55-- 2024-05-27T12:21:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 17s 2024-05-27T12:21:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 340M in 16s (21.1 MB/s) 2024-05-27T12:21:55.545 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-27T12:21:55.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T12:21:57.641 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying... ######################################## 2024-05-27T12:21:57.866 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing... ######################################## 2024-05-27T12:21:57.978 INFO:teuthology.orchestra.run.smithi094.stdout:Updating / installing... 2024-05-27T12:22:05.995 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T12:22:24.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27 12:22:24 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T12:22:24.376 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-05-27 12:22:24-- 2024-05-27T12:22:24.376 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 29s 2024-05-27T12:22:24.376 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 340M in 29s (11.8 MB/s) 2024-05-27T12:22:24.379 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.1... 2024-05-27T12:22:24.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T12:22:26.462 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2024-05-27T12:22:26.592 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2024-05-27T12:22:26.684 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2024-05-27T12:22:34.580 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T12:22:46.048 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T12:22:46.048 DEBUG:teuthology.orchestra.run.smithi094:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T12:22:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T12:22:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T12:22:46.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T12:22:46.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T12:22:46.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T12:22:46.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T12:22:46.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T12:22:46.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T12:22:46.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T12:22:46.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T12:22:46.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T12:22:46.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T12:22:46.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T12:22:46.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T12:22:46.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T12:22:46.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T12:22:46.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T12:22:46.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T12:22:46.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T12:22:46.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T12:22:46.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T12:22:46.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T12:22:46.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T12:22:46.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T12:22:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T12:22:46.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T12:22:46.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T12:22:46.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T12:22:46.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T12:22:46.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T12:22:46.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T12:22:46.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T12:22:46.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T12:22:46.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T12:22:46.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T12:22:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T12:22:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T12:22:46.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T12:22:46.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T12:22:46.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T12:22:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T12:22:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T12:22:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T12:22:46.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T12:22:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T12:22:46.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T12:22:46.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T12:22:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T12:22:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T12:22:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T12:22:46.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T12:22:46.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T12:22:46.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T12:22:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T12:22:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T12:22:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T12:22:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T12:22:46.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T12:22:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T12:22:46.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T12:22:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T12:22:46.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T12:22:46.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T12:22:46.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T12:22:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T12:22:46.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T12:22:46.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T12:22:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T12:22:46.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T12:22:46.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T12:22:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T12:22:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T12:22:46.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T12:22:46.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T12:22:46.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T12:22:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T12:22:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T12:22:46.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T12:22:46.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T12:22:46.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T12:22:46.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T12:22:46.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T12:22:46.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T12:22:46.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T12:22:46.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T12:22:46.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T12:22:46.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T12:22:46.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T12:22:46.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T12:22:46.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T12:22:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T12:22:46.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T12:22:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T12:22:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T12:22:46.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T12:22:46.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T12:22:46.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T12:22:46.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T12:22:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T12:22:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T12:22:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T12:22:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T12:22:46.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T12:22:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T12:22:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T12:22:46.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T12:22:46.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T12:22:46.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T12:22:46.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T12:22:46.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T12:22:46.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T12:22:46.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T12:22:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T12:22:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T12:22:46.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T12:22:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T12:22:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T12:22:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T12:22:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T12:22:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T12:22:46.241 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T12:22:46.242 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T12:22:46.243 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T12:22:46.244 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T12:22:46.245 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T12:22:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T12:22:46.247 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T12:22:46.248 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T12:22:46.249 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T12:22:46.250 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T12:22:46.251 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T12:22:46.252 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T12:22:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T12:22:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T12:22:46.255 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T12:22:46.256 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T12:22:46.257 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T12:22:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T12:22:46.259 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T12:22:46.260 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T12:22:46.261 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T12:22:46.262 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T12:22:46.263 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T12:22:46.264 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T12:22:46.265 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T12:22:46.266 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T12:22:46.267 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T12:22:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T12:22:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T12:22:46.270 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T12:22:46.271 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T12:22:46.272 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T12:22:46.273 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T12:22:46.274 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T12:22:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T12:22:46.276 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T12:22:46.276 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T12:22:46.276 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T12:22:46.276 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T12:22:46.276 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T12:22:46.280 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T12:22:46.280 DEBUG:teuthology.orchestra.run.smithi094:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T12:22:46.302 INFO:teuthology.orchestra.run.smithi094.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:for file in vmlinuz System.map config; do 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: fi 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T12:22:46.303 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T12:22:46.304 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-05-27T12:22:46.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T12:22:46.373 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:22:46.374 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T12:22:46.375 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T12:22:46.375 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 6.9.0-gb9459a26b088 2024-05-27T12:22:46.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T12:22:46.912 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-05-27T12:22:48.303 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T12:22:48.337 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-05-27T12:22:48.339 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-05-27T12:22:48.359 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.UPWSTBfMt0 2024-05-27T12:22:48.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UPWSTBfMt0 2024-05-27T12:22:48.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.UPWSTBfMt0 2024-05-27T12:22:48.562 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.UPWSTBfMt0 is 6KB 2024-05-27T12:22:48.611 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.UPWSTBfMt0 2024-05-27T12:22:48.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T12:22:48.688 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T12:22:48.688 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T12:22:48.688 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T12:22:48.688 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T12:22:48.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T12:22:48.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-05-27T12:23:11.566 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T12:23:11.567 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T12:23:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T12:23:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T12:23:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T12:23:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T12:23:11.646 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T12:23:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T12:23:11.648 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T12:23:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T12:23:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T12:23:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T12:23:11.652 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T12:23:11.653 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T12:23:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T12:23:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T12:23:11.656 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T12:23:11.657 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T12:23:11.658 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T12:23:11.659 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T12:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T12:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T12:23:11.662 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T12:23:11.663 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T12:23:11.664 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T12:23:11.665 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T12:23:11.666 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T12:23:11.667 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T12:23:11.668 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T12:23:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T12:23:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T12:23:11.671 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T12:23:11.672 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T12:23:11.673 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T12:23:11.674 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T12:23:11.675 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T12:23:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T12:23:11.677 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T12:23:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T12:23:11.679 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T12:23:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T12:23:11.681 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T12:23:11.682 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T12:23:11.683 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T12:23:11.684 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T12:23:11.685 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T12:23:11.686 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T12:23:11.687 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T12:23:11.688 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T12:23:11.689 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T12:23:11.690 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T12:23:11.691 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T12:23:11.692 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T12:23:11.693 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T12:23:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T12:23:11.695 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T12:23:11.696 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T12:23:11.697 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T12:23:11.698 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T12:23:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T12:23:11.700 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T12:23:11.701 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T12:23:11.702 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T12:23:11.703 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T12:23:11.704 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T12:23:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T12:23:11.706 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T12:23:11.707 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T12:23:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T12:23:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T12:23:11.710 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T12:23:11.711 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T12:23:11.712 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T12:23:11.713 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T12:23:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T12:23:11.715 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T12:23:11.716 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T12:23:11.717 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T12:23:11.718 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T12:23:11.719 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T12:23:11.720 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T12:23:11.721 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T12:23:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T12:23:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T12:23:11.724 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T12:23:11.725 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T12:23:11.726 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T12:23:11.727 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T12:23:11.728 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T12:23:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T12:23:11.730 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T12:23:11.731 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T12:23:11.732 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T12:23:11.733 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T12:23:11.734 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T12:23:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T12:23:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T12:23:11.737 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T12:23:11.738 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T12:23:11.739 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T12:23:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T12:23:11.741 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T12:23:11.742 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T12:23:11.743 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T12:23:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T12:23:11.745 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T12:23:11.746 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T12:23:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T12:23:11.748 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T12:23:11.749 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T12:23:11.750 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T12:23:11.751 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T12:23:11.752 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T12:23:11.753 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T12:23:11.754 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T12:23:11.755 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T12:23:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T12:23:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T12:23:11.758 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T12:23:11.759 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T12:23:11.760 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T12:23:11.761 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T12:23:11.762 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T12:23:11.763 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T12:23:11.764 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T12:23:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T12:23:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T12:23:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T12:23:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T12:23:11.769 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T12:23:11.770 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T12:23:11.771 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T12:23:11.772 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T12:23:11.773 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T12:23:11.774 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T12:23:11.775 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T12:23:11.776 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T12:23:11.777 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T12:23:11.778 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T12:23:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T12:23:11.780 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T12:23:11.781 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T12:23:11.782 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T12:23:11.783 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T12:23:11.784 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T12:23:11.785 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T12:23:11.786 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T12:23:11.787 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T12:23:11.791 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T12:23:11.791 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T12:23:11.813 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T12:23:11.813 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T12:23:11.813 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:for file in vmlinuz System.map config; do 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: fi 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-05-27T12:23:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T12:23:11.815 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-05-27T12:23:11.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T12:23:11.888 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:23:11.889 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T12:23:11.890 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T12:23:11.891 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.9.0-gb9459a26b088 2024-05-27T12:23:11.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T12:23:12.437 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-05-27T12:23:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T12:23:13.857 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-05-27T12:23:13.859 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-05-27T12:23:13.873 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.yciSuYIgF4 2024-05-27T12:23:13.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yciSuYIgF4 2024-05-27T12:23:13.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.yciSuYIgF4 2024-05-27T12:23:14.080 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.yciSuYIgF4 is 6KB 2024-05-27T12:23:14.129 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.yciSuYIgF4 2024-05-27T12:23:14.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T12:23:14.205 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T12:23:14.205 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T12:23:14.206 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T12:23:14.206 INFO:teuthology.orchestra.run.smithi097.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T12:23:14.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T12:23:14.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-05-27T12:23:18.822 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-27T12:23:18.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-05-27T12:23:18.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:23:44.352 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-27T12:23:44.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:23:44.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:24:18.859 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T12:24:18.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-05-27T12:24:27.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-05-27T12:24:27.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:24:27.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:24:27.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:24:29.313 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-05-27T12:24:29.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-05-27T12:24:29.999 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-05-27T12:24:29.999 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-05-27T12:24:30.041 INFO:teuthology.orchestra.run.smithi094.stdout:6.9.0-gb9459a26b088 2024-05-27T12:24:30.041 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T12:24:30.042 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T12:24:30.042 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T12:24:31.045 DEBUG:teuthology.parallel:result is None 2024-05-27T12:24:46.370 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-05-27T12:24:58.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:24:58.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:25:01.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-05-27T12:25:16.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:25:16.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:25:16.876 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-05-27T12:25:17.618 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:25:17.619 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-gb9459a26b088"... 2024-05-27T12:25:17.619 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-05-27T12:25:17.699 INFO:teuthology.orchestra.run.smithi097.stdout:6.9.0-gb9459a26b088 2024-05-27T12:25:17.705 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T12:25:17.705 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T12:25:17.705 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T12:25:18.706 DEBUG:teuthology.parallel:result is None 2024-05-27T12:25:18.706 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T12:25:18.719 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T12:25:18.719 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T12:25:18.729 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T12:25:18.779 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T12:25:18.790 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T12:25:18.801 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T12:25:18.801 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T12:25:18.809 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T12:25:18.878 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T12:25:18.887 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T12:25:18.887 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T12:25:18.893 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T12:25:19.030 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T12:25:19.032 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T12:25:19.080 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T12:25:19.102 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T12:25:19.106 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T12:25:19.113 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T12:25:19.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T12:25:19.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T12:25:19.199 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T12:25:19.209 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T12:25:19.209 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T12:25:19.216 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T12:25:19.261 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T12:25:19.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T12:25:19.430 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T12:25:19.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T12:25:19.601 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:25:19.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T12:25:19.684 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T12:25:19.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T12:25:19.859 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T12:25:19.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T12:25:19.999 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:25:20.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T12:25:20.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-05-27T12:25:20.072 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-05-27T12:25:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T12:25:20.207 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T12:25:20.586 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T12:25:20.594 INFO:teuthology.task.internal:Starting timer... 2024-05-27T12:25:20.594 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T12:25:20.606 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T12:25:20.616 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-27T12:25:20.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-05-27T12:25:20.748 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-05-27T12:25:20.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-05-27T12:25:20.882 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-05-27T12:25:20.888 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T12:25:20.888 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-05-27T12:25:20.942 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-05-27T12:25:20.943 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-05-27T12:25:20.996 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-05-27T12:25:20.997 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive'} 2024-05-27T12:25:20.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T12:25:21.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:25:21.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T12:25:21.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:25:21.154 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T12:25:21.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-05-27T12:25:21.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-05-27T12:25:21.310 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T12:25:21.319 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T12:25:21.334 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T12:25:21.336 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi097.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T12:31:48.551 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com')] 2024-05-27T12:31:48.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-05-27T12:31:48.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:31:48.675 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-05-27T12:31:48.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-05-27T12:31:48.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:31:48.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T12:31:48.918 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-05-27T12:31:49.071 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-05-27T12:31:49.071 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T12:31:49.087 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T12:31:49.088 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T12:31:49.088 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T12:31:49.096 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T12:31:49.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T12:31:49.181 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T12:31:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T12:31:49.221 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T12:31:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T12:31:49.291 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-05-27T12:31:49.297 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-05-27T12:31:49.322 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-05-27T12:31:49.328 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-05-27T12:31:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T12:31:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T12:31:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T12:31:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T12:31:49.526 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-05-27T12:31:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-05-27T12:31:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T12:31:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-05-27T12:31:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.539 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T12:31:49.540 INFO:teuthology.run_tasks:Running task install... 2024-05-27T12:31:49.549 DEBUG:teuthology.task.install:project ceph 2024-05-27T12:31:49.549 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}} 2024-05-27T12:31:49.550 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'} 2024-05-27T12:31:49.550 INFO:teuthology.task.install:Using flavor: default 2024-05-27T12:31:49.555 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T12:31:49.555 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T12:31:49.556 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'tag': None, 'wait_for_package': False} 2024-05-27T12:31:49.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-27T12:31:49.558 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'tag': None, 'wait_for_package': False} 2024-05-27T12:31:49.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-27T12:31:49.702 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/ 2024-05-27T12:31:49.702 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g81797d87 2024-05-27T12:31:49.778 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/ 2024-05-27T12:31:49.778 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g81797d87 2024-05-27T12:31:49.820 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T12:31:49.820 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:31:49.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T12:31:49.888 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T12:31:49.888 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:31:49.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T12:31:49.912 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T12:31:49.912 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81797d879079e1a392b37a9c23a946594f658c50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T12:31:49.976 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T12:31:49.976 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81797d879079e1a392b37a9c23a946594f658c50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T12:31:50.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T12:31:50.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T12:31:50.155 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T12:31:50.215 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T12:31:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-05-27T12:31:50.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-05-27T12:31:50.294 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-05-27T12:31:50.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-05-27T12:31:50.566 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-05-27T12:31:50.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T12:31:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:45 files removed 2024-05-27T12:31:50.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T12:31:51.430 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-05-27T12:31:51.502 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 205 kB/s | 83 kB 00:00 2024-05-27T12:31:51.989 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-27T12:31:52.059 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-27T12:31:52.498 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-05-27T12:31:52.618 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-27T12:31:53.475 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-27T12:31:54.121 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-27T12:31:57.948 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-05-27T12:32:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-05-27T12:32:02.341 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-27T12:32:03.917 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 17 kB 00:00 2024-05-27T12:32:05.334 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-27T12:32:07.052 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 17 kB 00:00 2024-05-27T12:32:08.079 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-27T12:32:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 2.9 MB/s | 21 MB 00:07 2024-05-27T12:32:14.262 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-27T12:32:16.737 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T12:32:16.739 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T12:32:16.761 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T12:32:16.762 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T12:32:16.763 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T12:32:16.765 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T12:32:16.767 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T12:32:16.771 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T12:32:16.772 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T12:32:16.844 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-05-27T12:32:16.867 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-05-27T12:32:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-05-27T12:32:16.867 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-05-27T12:32:16.867 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3154.g81797d87.el9 ceph 6.6 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 890 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g81797d87.el9 ceph 145 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3154.g81797d87.el9 ceph 1.5 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 142 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 3.5 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 7.4 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 49 k 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 11 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3154.g81797d87.el9 ceph 49 M 2024-05-27T12:32:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 273 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 763 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 224 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3154.g81797d87.el9 ceph 27 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 18 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 32 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 747 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 127 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T12:32:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3154.g81797d87.el9 ceph 161 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3154.g81797d87.el9 ceph 321 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 298 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 99 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 85 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 170 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T12:32:16.870 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.4 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-05-27T12:32:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 21 M 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 34 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3154.g81797d87.el9 ceph 2.4 M 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 252 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3154.g81797d87.el9 ceph 4.7 M 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 17 M 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 16 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3154.g81797d87.el9 ceph 25 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T12:32:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3154.g81797d87.el9 ceph 163 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 501 k 2024-05-27T12:32:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T12:32:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 45 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 135 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T12:32:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T12:32:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T12:32:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T12:32:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T12:32:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T12:32:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T12:32:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 11 Packages 2024-05-27T12:32:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:32:16.884 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 226 M 2024-05-27T12:32:16.884 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-05-27T12:32:17.585 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-27T12:32:17.972 INFO:teuthology.orchestra.run.smithi097.stdout:(1/159): ceph-19.0.0-3154.g81797d87.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-05-27T12:32:18.456 INFO:teuthology.orchestra.run.smithi097.stdout:(2/159): ceph-fuse-19.0.0-3154.g81797d87.el9.x8 1.8 MB/s | 890 kB 00:00 2024-05-27T12:32:18.539 INFO:teuthology.orchestra.run.smithi097.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-05-27T12:32:19.181 INFO:teuthology.orchestra.run.smithi097.stdout:(4/159): ceph-common-19.0.0-3154.g81797d87.el9. 14 MB/s | 21 MB 00:01 2024-05-27T12:32:19.273 INFO:teuthology.orchestra.run.smithi097.stdout:(5/159): ceph-mds-19.0.0-3154.g81797d87.el9.x86 3.2 MB/s | 2.4 MB 00:00 2024-05-27T12:32:19.390 INFO:teuthology.orchestra.run.smithi097.stdout:(6/159): ceph-base-19.0.0-3154.g81797d87.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-27T12:32:19.448 INFO:teuthology.orchestra.run.smithi097.stdout:(7/159): ceph-mgr-19.0.0-3154.g81797d87.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-05-27T12:32:20.035 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T12:32:20.036 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T12:32:20.054 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T12:32:20.055 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T12:32:20.057 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T12:32:20.058 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T12:32:20.060 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T12:32:20.063 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T12:32:20.064 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T12:32:20.135 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T12:32:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3154.g81797d87.el9 ceph 6.6 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 890 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g81797d87.el9 ceph 145 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3154.g81797d87.el9 ceph 1.5 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 142 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 3.5 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 7.4 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 49 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 11 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3154.g81797d87.el9 ceph 49 M 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 273 k 2024-05-27T12:32:20.162 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 763 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 224 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3154.g81797d87.el9 ceph 27 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 18 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 32 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 747 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 127 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T12:32:20.163 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3154.g81797d87.el9 ceph 161 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3154.g81797d87.el9 ceph 321 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 298 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 99 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 85 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 170 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T12:32:20.164 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.4 M 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 21 M 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 34 k 2024-05-27T12:32:20.165 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3154.g81797d87.el9 ceph 2.4 M 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 252 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3154.g81797d87.el9 ceph 4.7 M 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 17 M 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 16 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3154.g81797d87.el9 ceph 25 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T12:32:20.166 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3154.g81797d87.el9 ceph 163 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 501 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T12:32:20.167 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T12:32:20.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 45 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 135 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T12:32:20.169 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T12:32:20.170 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T12:32:20.171 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T12:32:20.172 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T12:32:20.173 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 11 Packages 2024-05-27T12:32:20.174 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:32:20.175 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 226 M 2024-05-27T12:32:20.175 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-05-27T12:32:20.299 INFO:teuthology.orchestra.run.smithi097.stdout:(8/159): ceph-radosgw-19.0.0-3154.g81797d87.el9 12 MB/s | 11 MB 00:00 2024-05-27T12:32:20.407 INFO:teuthology.orchestra.run.smithi097.stdout:(9/159): ceph-mon-19.0.0-3154.g81797d87.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-05-27T12:32:20.424 INFO:teuthology.orchestra.run.smithi097.stdout:(10/159): ceph-selinux-19.0.0-3154.g81797d87.el 203 kB/s | 25 kB 00:00 2024-05-27T12:32:20.499 INFO:teuthology.orchestra.run.smithi097.stdout:(11/159): cephfs-mirror-19.0.0-3154.g81797d87.e 3.0 MB/s | 224 kB 00:00 2024-05-27T12:32:20.558 INFO:teuthology.orchestra.run.smithi097.stdout:(12/159): cephfs-shell-19.0.0-3154.g81797d87.el 469 kB/s | 27 kB 00:00 2024-05-27T12:32:20.616 INFO:teuthology.orchestra.run.smithi097.stdout:(13/159): libcephfs-devel-19.0.0-3154.g81797d87 552 kB/s | 32 kB 00:00 2024-05-27T12:32:20.725 INFO:teuthology.orchestra.run.smithi097.stdout:(14/159): libcephfs2-19.0.0-3154.g81797d87.el9. 6.8 MB/s | 747 kB 00:00 2024-05-27T12:32:20.809 INFO:teuthology.orchestra.run.smithi097.stdout:(15/159): libcephsqlite-19.0.0-3154.g81797d87.e 1.9 MB/s | 163 kB 00:00 2024-05-27T12:32:20.876 INFO:teuthology.orchestra.run.smithi097.stdout:(16/159): librados-devel-19.0.0-3154.g81797d87. 1.9 MB/s | 127 kB 00:00 2024-05-27T12:32:21.046 INFO:teuthology.orchestra.run.smithi094.stdout:(1/159): ceph-19.0.0-3154.g81797d87.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-27T12:32:21.151 INFO:teuthology.orchestra.run.smithi097.stdout:(17/159): ceph-osd-19.0.0-3154.g81797d87.el9.x8 9.6 MB/s | 17 MB 00:01 2024-05-27T12:32:21.193 INFO:teuthology.orchestra.run.smithi097.stdout:(18/159): libradosstriper1-19.0.0-3154.g81797d8 1.5 MB/s | 501 kB 00:00 2024-05-27T12:32:21.260 INFO:teuthology.orchestra.run.smithi097.stdout:(19/159): python3-ceph-argparse-19.0.0-3154.g81 685 kB/s | 45 kB 00:00 2024-05-27T12:32:21.329 INFO:teuthology.orchestra.run.smithi094.stdout:(2/159): ceph-fuse-19.0.0-3154.g81797d87.el9.x8 3.1 MB/s | 890 kB 00:00 2024-05-27T12:32:21.335 INFO:teuthology.orchestra.run.smithi097.stdout:(20/159): python3-ceph-common-19.0.0-3154.g8179 1.8 MB/s | 135 kB 00:00 2024-05-27T12:32:21.398 INFO:teuthology.orchestra.run.smithi094.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 2.2 MB/s | 145 kB 00:00 2024-05-27T12:32:21.419 INFO:teuthology.orchestra.run.smithi097.stdout:(21/159): python3-cephfs-19.0.0-3154.g81797d87. 1.9 MB/s | 161 kB 00:00 2024-05-27T12:32:21.511 INFO:teuthology.orchestra.run.smithi097.stdout:(22/159): python3-rados-19.0.0-3154.g81797d87.e 3.5 MB/s | 321 kB 00:00 2024-05-27T12:32:21.628 INFO:teuthology.orchestra.run.smithi097.stdout:(23/159): python3-rbd-19.0.0-3154.g81797d87.el9 2.5 MB/s | 298 kB 00:00 2024-05-27T12:32:21.720 INFO:teuthology.orchestra.run.smithi097.stdout:(24/159): python3-rgw-19.0.0-3154.g81797d87.el9 1.1 MB/s | 99 kB 00:00 2024-05-27T12:32:21.778 INFO:teuthology.orchestra.run.smithi097.stdout:(25/159): rbd-fuse-19.0.0-3154.g81797d87.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-27T12:32:21.805 INFO:teuthology.orchestra.run.smithi094.stdout:(4/159): ceph-mds-19.0.0-3154.g81797d87.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-05-27T12:32:22.113 INFO:teuthology.orchestra.run.smithi094.stdout:(5/159): ceph-mgr-19.0.0-3154.g81797d87.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-05-27T12:32:22.137 INFO:teuthology.orchestra.run.smithi097.stdout:(26/159): rbd-mirror-19.0.0-3154.g81797d87.el9. 8.6 MB/s | 3.1 MB 00:00 2024-05-27T12:32:22.196 INFO:teuthology.orchestra.run.smithi097.stdout:(27/159): rbd-nbd-19.0.0-3154.g81797d87.el9.x86 2.9 MB/s | 170 kB 00:00 2024-05-27T12:32:22.254 INFO:teuthology.orchestra.run.smithi097.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 594 kB/s | 34 kB 00:00 2024-05-27T12:32:22.305 INFO:teuthology.orchestra.run.smithi097.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g81797d8 2.8 MB/s | 142 kB 00:00 2024-05-27T12:32:22.460 INFO:teuthology.orchestra.run.smithi097.stdout:(30/159): librgw2-19.0.0-3154.g81797d87.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-05-27T12:32:22.564 INFO:teuthology.orchestra.run.smithi094.stdout:(6/159): ceph-mon-19.0.0-3154.g81797d87.el9.x86 10 MB/s | 4.7 MB 00:00 2024-05-27T12:32:22.664 INFO:teuthology.orchestra.run.smithi097.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3154.g81797 9.9 MB/s | 3.5 MB 00:00 2024-05-27T12:32:22.722 INFO:teuthology.orchestra.run.smithi097.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3154.g81 4.3 MB/s | 252 kB 00:00 2024-05-27T12:32:22.781 INFO:teuthology.orchestra.run.smithi097.stdout:(33/159): ceph-mgr-rook-19.0.0-3154.g81797d87.e 856 kB/s | 49 kB 00:00 2024-05-27T12:32:22.839 INFO:teuthology.orchestra.run.smithi097.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3154.g8 284 kB/s | 16 kB 00:00 2024-05-27T12:32:22.923 INFO:teuthology.orchestra.run.smithi097.stdout:(35/159): ceph-volume-19.0.0-3154.g81797d87.el9 3.2 MB/s | 273 kB 00:00 2024-05-27T12:32:23.073 INFO:teuthology.orchestra.run.smithi097.stdout:(36/159): cephadm-19.0.0-3154.g81797d87.el9.noa 5.0 MB/s | 763 kB 00:00 2024-05-27T12:32:23.132 INFO:teuthology.orchestra.run.smithi097.stdout:(37/159): cephfs-top-19.0.0-3154.g81797d87.el9. 314 kB/s | 18 kB 00:00 2024-05-27T12:32:23.189 INFO:teuthology.orchestra.run.smithi094.stdout:(7/159): ceph-base-19.0.0-3154.g81797d87.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-05-27T12:32:23.891 INFO:teuthology.orchestra.run.smithi097.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 106 kB/s | 80 kB 00:00 2024-05-27T12:32:24.294 INFO:teuthology.orchestra.run.smithi097.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-05-27T12:32:24.489 INFO:teuthology.orchestra.run.smithi094.stdout:(8/159): ceph-osd-19.0.0-3154.g81797d87.el9.x86 8.7 MB/s | 17 MB 00:01 2024-05-27T12:32:24.540 INFO:teuthology.orchestra.run.smithi094.stdout:(9/159): ceph-selinux-19.0.0-3154.g81797d87.el9 514 kB/s | 25 kB 00:00 2024-05-27T12:32:24.723 INFO:teuthology.orchestra.run.smithi094.stdout:(10/159): ceph-radosgw-19.0.0-3154.g81797d87.el 6.9 MB/s | 11 MB 00:01 2024-05-27T12:32:24.816 INFO:teuthology.orchestra.run.smithi094.stdout:(11/159): cephfs-mirror-19.0.0-3154.g81797d87.e 2.4 MB/s | 224 kB 00:00 2024-05-27T12:32:24.916 INFO:teuthology.orchestra.run.smithi094.stdout:(12/159): cephfs-shell-19.0.0-3154.g81797d87.el 273 kB/s | 27 kB 00:00 2024-05-27T12:32:24.958 INFO:teuthology.orchestra.run.smithi097.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 754 kB/s | 803 kB 00:01 2024-05-27T12:32:24.991 INFO:teuthology.orchestra.run.smithi094.stdout:(13/159): libcephfs-devel-19.0.0-3154.g81797d87 427 kB/s | 32 kB 00:00 2024-05-27T12:32:25.125 INFO:teuthology.orchestra.run.smithi097.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 235 kB/s | 195 kB 00:00 2024-05-27T12:32:25.133 INFO:teuthology.orchestra.run.smithi094.stdout:(14/159): libcephfs2-19.0.0-3154.g81797d87.el9. 5.2 MB/s | 747 kB 00:00 2024-05-27T12:32:25.226 INFO:teuthology.orchestra.run.smithi094.stdout:(15/159): libcephsqlite-19.0.0-3154.g81797d87.e 1.8 MB/s | 163 kB 00:00 2024-05-27T12:32:25.301 INFO:teuthology.orchestra.run.smithi094.stdout:(16/159): librados-devel-19.0.0-3154.g81797d87. 1.7 MB/s | 127 kB 00:00 2024-05-27T12:32:25.401 INFO:teuthology.orchestra.run.smithi094.stdout:(17/159): libradosstriper1-19.0.0-3154.g81797d8 4.9 MB/s | 501 kB 00:00 2024-05-27T12:32:25.834 INFO:teuthology.orchestra.run.smithi097.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 290 kB/s | 253 kB 00:00 2024-05-27T12:32:25.985 INFO:teuthology.orchestra.run.smithi094.stdout:(18/159): librgw2-19.0.0-3154.g81797d87.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-05-27T12:32:26.060 INFO:teuthology.orchestra.run.smithi094.stdout:(19/159): python3-ceph-argparse-19.0.0-3154.g81 610 kB/s | 45 kB 00:00 2024-05-27T12:32:26.136 INFO:teuthology.orchestra.run.smithi094.stdout:(20/159): python3-ceph-common-19.0.0-3154.g8179 1.8 MB/s | 135 kB 00:00 2024-05-27T12:32:26.220 INFO:teuthology.orchestra.run.smithi094.stdout:(21/159): python3-cephfs-19.0.0-3154.g81797d87. 1.9 MB/s | 161 kB 00:00 2024-05-27T12:32:26.276 INFO:teuthology.orchestra.run.smithi097.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-27T12:32:26.320 INFO:teuthology.orchestra.run.smithi094.stdout:(22/159): python3-rados-19.0.0-3154.g81797d87.e 3.2 MB/s | 321 kB 00:00 2024-05-27T12:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout:(23/159): python3-rbd-19.0.0-3154.g81797d87.el9 2.5 MB/s | 298 kB 00:00 2024-05-27T12:32:26.537 INFO:teuthology.orchestra.run.smithi094.stdout:(24/159): python3-rgw-19.0.0-3154.g81797d87.el9 1.0 MB/s | 99 kB 00:00 2024-05-27T12:32:26.629 INFO:teuthology.orchestra.run.smithi094.stdout:(25/159): rbd-fuse-19.0.0-3154.g81797d87.el9.x8 937 kB/s | 85 kB 00:00 2024-05-27T12:32:26.676 INFO:teuthology.orchestra.run.smithi097.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 127 kB/s | 106 kB 00:00 2024-05-27T12:32:27.038 INFO:teuthology.orchestra.run.smithi094.stdout:(26/159): rbd-mirror-19.0.0-3154.g81797d87.el9. 7.6 MB/s | 3.1 MB 00:00 2024-05-27T12:32:27.147 INFO:teuthology.orchestra.run.smithi094.stdout:(27/159): rbd-nbd-19.0.0-3154.g81797d87.el9.x86 1.5 MB/s | 170 kB 00:00 2024-05-27T12:32:27.255 INFO:teuthology.orchestra.run.smithi094.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 318 kB/s | 34 kB 00:00 2024-05-27T12:32:27.318 INFO:teuthology.orchestra.run.smithi097.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 130 kB/s | 135 kB 00:01 2024-05-27T12:32:27.330 INFO:teuthology.orchestra.run.smithi094.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g81797d8 1.9 MB/s | 142 kB 00:00 2024-05-27T12:32:27.664 INFO:teuthology.orchestra.run.smithi094.stdout:(30/159): ceph-mgr-dashboard-19.0.0-3154.g81797 11 MB/s | 3.5 MB 00:00 2024-05-27T12:32:27.968 INFO:teuthology.orchestra.run.smithi097.stdout:(46/159): ceph-test-19.0.0-3154.g81797d87.el9.x 6.5 MB/s | 49 MB 00:07 2024-05-27T12:32:28.001 INFO:teuthology.orchestra.run.smithi097.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 94 kB/s | 125 kB 00:01 2024-05-27T12:32:28.194 INFO:teuthology.orchestra.run.smithi097.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-27T12:32:28.223 INFO:teuthology.orchestra.run.smithi094.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-05-27T12:32:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-27T12:32:28.286 INFO:teuthology.orchestra.run.smithi097.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-05-27T12:32:28.298 INFO:teuthology.orchestra.run.smithi094.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3154.g81 3.3 MB/s | 252 kB 00:00 2024-05-27T12:32:28.311 INFO:teuthology.orchestra.run.smithi097.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-05-27T12:32:28.345 INFO:teuthology.orchestra.run.smithi097.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-27T12:32:28.370 INFO:teuthology.orchestra.run.smithi097.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-05-27T12:32:28.399 INFO:teuthology.orchestra.run.smithi094.stdout:(33/159): ceph-mgr-rook-19.0.0-3154.g81797d87.e 498 kB/s | 49 kB 00:00 2024-05-27T12:32:28.412 INFO:teuthology.orchestra.run.smithi097.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 445 kB/s | 18 kB 00:00 2024-05-27T12:32:28.446 INFO:teuthology.orchestra.run.smithi097.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 344 kB/s | 11 kB 00:00 2024-05-27T12:32:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-27T12:32:28.491 INFO:teuthology.orchestra.run.smithi094.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3154.g8 181 kB/s | 16 kB 00:00 2024-05-27T12:32:28.554 INFO:teuthology.orchestra.run.smithi097.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-27T12:32:28.580 INFO:teuthology.orchestra.run.smithi097.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 495 kB/s | 45 kB 00:00 2024-05-27T12:32:28.630 INFO:teuthology.orchestra.run.smithi097.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-27T12:32:28.689 INFO:teuthology.orchestra.run.smithi097.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-05-27T12:32:28.705 INFO:teuthology.orchestra.run.smithi097.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-05-27T12:32:28.739 INFO:teuthology.orchestra.run.smithi097.stdout:(62/159): python3-urllib3-1.26.5-5.el9.noarch.r 151 kB/s | 215 kB 00:01 2024-05-27T12:32:28.764 INFO:teuthology.orchestra.run.smithi097.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 636 kB/s | 47 kB 00:00 2024-05-27T12:32:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-27T12:32:28.840 INFO:teuthology.orchestra.run.smithi097.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-27T12:32:28.865 INFO:teuthology.orchestra.run.smithi097.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 358 kB/s | 35 kB 00:00 2024-05-27T12:32:29.015 INFO:teuthology.orchestra.run.smithi097.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-27T12:32:29.025 INFO:teuthology.orchestra.run.smithi094.stdout:(35/159): ceph-test-19.0.0-3154.g81797d87.el9.x 11 MB/s | 49 MB 00:04 2024-05-27T12:32:29.058 INFO:teuthology.orchestra.run.smithi094.stdout:(36/159): ceph-volume-19.0.0-3154.g81797d87.el9 482 kB/s | 273 kB 00:00 2024-05-27T12:32:29.125 INFO:teuthology.orchestra.run.smithi094.stdout:(37/159): cephadm-19.0.0-3154.g81797d87.el9.noa 7.5 MB/s | 763 kB 00:00 2024-05-27T12:32:29.141 INFO:teuthology.orchestra.run.smithi097.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-05-27T12:32:29.151 INFO:teuthology.orchestra.run.smithi094.stdout:(38/159): cephfs-top-19.0.0-3154.g81797d87.el9. 200 kB/s | 18 kB 00:00 2024-05-27T12:32:29.208 INFO:teuthology.orchestra.run.smithi097.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-05-27T12:32:29.233 INFO:teuthology.orchestra.run.smithi097.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-27T12:32:29.275 INFO:teuthology.orchestra.run.smithi097.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-05-27T12:32:29.325 INFO:teuthology.orchestra.run.smithi097.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-27T12:32:29.375 INFO:teuthology.orchestra.run.smithi097.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-05-27T12:32:29.417 INFO:teuthology.orchestra.run.smithi097.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T12:32:29.467 INFO:teuthology.orchestra.run.smithi097.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-27T12:32:29.518 INFO:teuthology.orchestra.run.smithi097.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T12:32:29.586 INFO:teuthology.orchestra.run.smithi097.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-05-27T12:32:29.668 INFO:teuthology.orchestra.run.smithi097.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-05-27T12:32:29.719 INFO:teuthology.orchestra.run.smithi097.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-27T12:32:29.836 INFO:teuthology.orchestra.run.smithi097.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 9.6 MB/s | 5.8 MB 00:00 2024-05-27T12:32:29.861 INFO:teuthology.orchestra.run.smithi097.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 246 kB/s | 35 kB 00:00 2024-05-27T12:32:29.911 INFO:teuthology.orchestra.run.smithi097.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-27T12:32:29.968 INFO:teuthology.orchestra.run.smithi094.stdout:(39/159): ceph-common-19.0.0-3154.g81797d87.el9 2.3 MB/s | 21 MB 00:09 2024-05-27T12:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 92 kB/s | 80 kB 00:00 2024-05-27T12:32:30.062 INFO:teuthology.orchestra.run.smithi097.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 8.1 MB/s | 6.0 MB 00:00 2024-05-27T12:32:30.095 INFO:teuthology.orchestra.run.smithi097.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-05-27T12:32:30.121 INFO:teuthology.orchestra.run.smithi097.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T12:32:30.146 INFO:teuthology.orchestra.run.smithi097.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 855 kB/s | 42 kB 00:00 2024-05-27T12:32:30.179 INFO:teuthology.orchestra.run.smithi097.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-27T12:32:30.213 INFO:teuthology.orchestra.run.smithi097.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-27T12:32:30.255 INFO:teuthology.orchestra.run.smithi097.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-27T12:32:30.280 INFO:teuthology.orchestra.run.smithi097.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-05-27T12:32:30.306 INFO:teuthology.orchestra.run.smithi097.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T12:32:30.348 INFO:teuthology.orchestra.run.smithi097.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T12:32:30.390 INFO:teuthology.orchestra.run.smithi097.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T12:32:30.460 INFO:teuthology.orchestra.run.smithi094.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 614 kB/s | 803 kB 00:01 2024-05-27T12:32:30.465 INFO:teuthology.orchestra.run.smithi097.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-27T12:32:30.574 INFO:teuthology.orchestra.run.smithi097.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.9 MB/s | 5.1 MB 00:00 2024-05-27T12:32:30.599 INFO:teuthology.orchestra.run.smithi097.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-05-27T12:32:30.624 INFO:teuthology.orchestra.run.smithi097.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-05-27T12:32:30.658 INFO:teuthology.orchestra.run.smithi097.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-27T12:32:30.683 INFO:teuthology.orchestra.run.smithi097.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-27T12:32:30.708 INFO:teuthology.orchestra.run.smithi097.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-05-27T12:32:30.734 INFO:teuthology.orchestra.run.smithi097.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T12:32:30.759 INFO:teuthology.orchestra.run.smithi097.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-05-27T12:32:30.869 INFO:teuthology.orchestra.run.smithi094.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 217 kB/s | 195 kB 00:00 2024-05-27T12:32:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 271 kB/s | 253 kB 00:00 2024-05-27T12:32:31.042 INFO:teuthology.orchestra.run.smithi097.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-27T12:32:31.068 INFO:teuthology.orchestra.run.smithi097.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-05-27T12:32:31.570 INFO:teuthology.orchestra.run.smithi094.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-27T12:32:31.637 INFO:teuthology.orchestra.run.smithi094.stdout:(45/159): python3-ply-3.11-14.el9.noarch.rpm 139 kB/s | 106 kB 00:00 2024-05-27T12:32:31.712 INFO:teuthology.orchestra.run.smithi094.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-05-27T12:32:31.793 INFO:teuthology.orchestra.run.smithi097.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-05-27T12:32:31.826 INFO:teuthology.orchestra.run.smithi097.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 408 kB/s | 309 kB 00:00 2024-05-27T12:32:31.894 INFO:teuthology.orchestra.run.smithi097.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-05-27T12:32:31.979 INFO:teuthology.orchestra.run.smithi094.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-27T12:32:32.002 INFO:teuthology.orchestra.run.smithi097.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-05-27T12:32:32.079 INFO:teuthology.orchestra.run.smithi094.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 9.3 MB/s | 943 kB 00:00 2024-05-27T12:32:32.119 INFO:teuthology.orchestra.run.smithi097.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-05-27T12:32:32.121 INFO:teuthology.orchestra.run.smithi094.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-27T12:32:32.155 INFO:teuthology.orchestra.run.smithi094.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 661 kB/s | 22 kB 00:00 2024-05-27T12:32:32.205 INFO:teuthology.orchestra.run.smithi094.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-05-27T12:32:32.228 INFO:teuthology.orchestra.run.smithi097.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 627 kB/s | 67 kB 00:00 2024-05-27T12:32:32.239 INFO:teuthology.orchestra.run.smithi094.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 1.0 MB/s | 33 kB 00:00 2024-05-27T12:32:32.345 INFO:teuthology.orchestra.run.smithi097.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-05-27T12:32:32.379 INFO:teuthology.orchestra.run.smithi097.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-27T12:32:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-27T12:32:32.415 INFO:teuthology.orchestra.run.smithi094.stdout:(54/159): python3-requests-2.25.1-8.el9.noarch. 148 kB/s | 125 kB 00:00 2024-05-27T12:32:32.440 INFO:teuthology.orchestra.run.smithi094.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-05-27T12:32:32.462 INFO:teuthology.orchestra.run.smithi097.stdout:(113/159): protobuf-compiler-3.14.0-13.el9.x86_ 507 kB/s | 863 kB 00:01 2024-05-27T12:32:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:(56/159): python3-urllib3-1.26.5-5.el9.noarch.r 254 kB/s | 215 kB 00:00 2024-05-27T12:32:32.507 INFO:teuthology.orchestra.run.smithi094.stdout:(57/159): libaio-devel-0.3.111-13.el9.x86_64.rp 122 kB/s | 11 kB 00:00 2024-05-27T12:32:32.521 INFO:teuthology.orchestra.run.smithi097.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-27T12:32:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-27T12:32:32.554 INFO:teuthology.orchestra.run.smithi097.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-27T12:32:32.566 INFO:teuthology.orchestra.run.smithi094.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-05-27T12:32:32.608 INFO:teuthology.orchestra.run.smithi094.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-05-27T12:32:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 384 kB/s | 25 kB 00:00 2024-05-27T12:32:32.657 INFO:teuthology.orchestra.run.smithi097.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 238 kB/s | 32 kB 00:00 2024-05-27T12:32:32.659 INFO:teuthology.orchestra.run.smithi094.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 957 kB/s | 47 kB 00:00 2024-05-27T12:32:32.696 INFO:teuthology.orchestra.run.smithi097.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 154 kB/s | 15 kB 00:00 2024-05-27T12:32:32.709 INFO:teuthology.orchestra.run.smithi094.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-27T12:32:32.743 INFO:teuthology.orchestra.run.smithi094.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-27T12:32:32.764 INFO:teuthology.orchestra.run.smithi097.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-05-27T12:32:32.768 INFO:teuthology.orchestra.run.smithi094.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 35 kB 00:00 2024-05-27T12:32:32.811 INFO:teuthology.orchestra.run.smithi094.stdout:(66/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-05-27T12:32:32.830 INFO:teuthology.orchestra.run.smithi097.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 2.7 MB/s | 358 kB 00:00 2024-05-27T12:32:32.880 INFO:teuthology.orchestra.run.smithi097.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-27T12:32:32.911 INFO:teuthology.orchestra.run.smithi094.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-27T12:32:32.931 INFO:teuthology.orchestra.run.smithi097.stdout:(121/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 93 kB/s | 43 kB 00:00 2024-05-27T12:32:32.964 INFO:teuthology.orchestra.run.smithi097.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-27T12:32:32.986 INFO:teuthology.orchestra.run.smithi094.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-27T12:32:33.077 INFO:teuthology.orchestra.run.smithi097.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-27T12:32:33.095 INFO:teuthology.orchestra.run.smithi094.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-27T12:32:33.107 INFO:teuthology.orchestra.run.smithi097.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 76 kB/s | 11 kB 00:00 2024-05-27T12:32:33.174 INFO:teuthology.orchestra.run.smithi097.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 185 kB/s | 18 kB 00:00 2024-05-27T12:32:33.195 INFO:teuthology.orchestra.run.smithi094.stdout:(70/159): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-05-27T12:32:33.207 INFO:teuthology.orchestra.run.smithi097.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-05-27T12:32:33.221 INFO:teuthology.orchestra.run.smithi094.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 571 kB/s | 133 kB 00:00 2024-05-27T12:32:33.249 INFO:teuthology.orchestra.run.smithi097.stdout:(127/159): python3-grpcio-tools-1.46.7-10.el9.x 455 kB/s | 144 kB 00:00 2024-05-27T12:32:33.263 INFO:teuthology.orchestra.run.smithi094.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-27T12:32:33.283 INFO:teuthology.orchestra.run.smithi097.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-27T12:32:33.304 INFO:teuthology.orchestra.run.smithi094.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-27T12:32:33.308 INFO:teuthology.orchestra.run.smithi097.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-05-27T12:32:33.347 INFO:teuthology.orchestra.run.smithi094.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T12:32:33.383 INFO:teuthology.orchestra.run.smithi097.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 465 kB/s | 46 kB 00:00 2024-05-27T12:32:33.397 INFO:teuthology.orchestra.run.smithi094.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-27T12:32:33.409 INFO:teuthology.orchestra.run.smithi097.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 791 kB/s | 79 kB 00:00 2024-05-27T12:32:33.448 INFO:teuthology.orchestra.run.smithi094.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T12:32:33.484 INFO:teuthology.orchestra.run.smithi097.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-05-27T12:32:33.489 INFO:teuthology.orchestra.run.smithi094.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 626 kB/s | 26 kB 00:00 2024-05-27T12:32:33.524 INFO:teuthology.orchestra.run.smithi094.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 583 kB/s | 19 kB 00:00 2024-05-27T12:32:33.534 INFO:teuthology.orchestra.run.smithi097.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-27T12:32:33.585 INFO:teuthology.orchestra.run.smithi097.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-05-27T12:32:33.641 INFO:teuthology.orchestra.run.smithi094.stdout:(79/159): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-05-27T12:32:33.643 INFO:teuthology.orchestra.run.smithi097.stdout:(135/159): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-05-27T12:32:33.668 INFO:teuthology.orchestra.run.smithi097.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2024-05-27T12:32:33.674 INFO:teuthology.orchestra.run.smithi094.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-27T12:32:33.691 INFO:teuthology.orchestra.run.smithi094.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 702 kB/s | 35 kB 00:00 2024-05-27T12:32:33.694 INFO:teuthology.orchestra.run.smithi097.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-05-27T12:32:33.742 INFO:teuthology.orchestra.run.smithi094.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-27T12:32:33.744 INFO:teuthology.orchestra.run.smithi097.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-05-27T12:32:33.777 INFO:teuthology.orchestra.run.smithi097.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-27T12:32:33.803 INFO:teuthology.orchestra.run.smithi097.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-05-27T12:32:33.845 INFO:teuthology.orchestra.run.smithi097.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-05-27T12:32:33.879 INFO:teuthology.orchestra.run.smithi097.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-05-27T12:32:33.884 INFO:teuthology.orchestra.run.smithi094.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.6 MB/s | 6.0 MB 00:00 2024-05-27T12:32:33.917 INFO:teuthology.orchestra.run.smithi094.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-05-27T12:32:33.920 INFO:teuthology.orchestra.run.smithi097.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-27T12:32:33.942 INFO:teuthology.orchestra.run.smithi094.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T12:32:33.968 INFO:teuthology.orchestra.run.smithi094.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 853 kB/s | 42 kB 00:00 2024-05-27T12:32:33.979 INFO:teuthology.orchestra.run.smithi097.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 673 kB/s | 90 kB 00:00 2024-05-27T12:32:34.002 INFO:teuthology.orchestra.run.smithi094.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-27T12:32:34.013 INFO:teuthology.orchestra.run.smithi097.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-27T12:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-27T12:32:34.038 INFO:teuthology.orchestra.run.smithi097.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-05-27T12:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-05-27T12:32:34.097 INFO:teuthology.orchestra.run.smithi097.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-27T12:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 771 kB/s | 51 kB 00:00 2024-05-27T12:32:34.128 INFO:teuthology.orchestra.run.smithi094.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T12:32:34.170 INFO:teuthology.orchestra.run.smithi094.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T12:32:34.180 INFO:teuthology.orchestra.run.smithi097.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-27T12:32:34.212 INFO:teuthology.orchestra.run.smithi094.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T12:32:34.263 INFO:teuthology.orchestra.run.smithi094.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-27T12:32:34.371 INFO:teuthology.orchestra.run.smithi094.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-05-27T12:32:34.396 INFO:teuthology.orchestra.run.smithi094.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-05-27T12:32:34.422 INFO:teuthology.orchestra.run.smithi094.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 958 kB/s | 47 kB 00:00 2024-05-27T12:32:34.456 INFO:teuthology.orchestra.run.smithi094.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-05-27T12:32:34.489 INFO:teuthology.orchestra.run.smithi094.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 874 kB/s | 58 kB 00:00 2024-05-27T12:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-05-27T12:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-27T12:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-05-27T12:32:34.597 INFO:teuthology.orchestra.run.smithi097.stdout:(149/159): librbd1-19.0.0-3154.g81797d87.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-27T12:32:34.689 INFO:teuthology.orchestra.run.smithi097.stdout:(150/159): librados2-19.0.0-3154.g81797d87.el9. 5.2 MB/s | 3.4 MB 00:00 2024-05-27T12:32:34.691 INFO:teuthology.orchestra.run.smithi094.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-05-27T12:32:34.900 INFO:teuthology.orchestra.run.smithi094.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-05-27T12:32:34.983 INFO:teuthology.orchestra.run.smithi094.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-05-27T12:32:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 138 kB/s | 19 kB 00:00 2024-05-27T12:32:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-27T12:32:35.193 INFO:teuthology.orchestra.run.smithi094.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-27T12:32:35.226 INFO:teuthology.orchestra.run.smithi094.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-05-27T12:32:35.252 INFO:teuthology.orchestra.run.smithi094.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-05-27T12:32:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-27T12:32:35.327 INFO:teuthology.orchestra.run.smithi094.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-27T12:32:35.411 INFO:teuthology.orchestra.run.smithi094.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-05-27T12:32:35.431 INFO:teuthology.orchestra.run.smithi097.stdout:(151/159): NetworkManager-1.47.90-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-05-27T12:32:35.453 INFO:teuthology.orchestra.run.smithi094.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-05-27T12:32:35.456 INFO:teuthology.orchestra.run.smithi097.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 54 kB/s | 41 kB 00:00 2024-05-27T12:32:35.487 INFO:teuthology.orchestra.run.smithi094.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 583 kB/s | 43 kB 00:00 2024-05-27T12:32:35.522 INFO:teuthology.orchestra.run.smithi094.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-05-27T12:32:35.546 INFO:teuthology.orchestra.run.smithi094.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 258 kB/s | 15 kB 00:00 2024-05-27T12:32:35.588 INFO:teuthology.orchestra.run.smithi094.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-27T12:32:35.648 INFO:teuthology.orchestra.run.smithi094.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-27T12:32:35.688 INFO:teuthology.orchestra.run.smithi094.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-27T12:32:35.722 INFO:teuthology.orchestra.run.smithi094.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.8 MB/s | 206 kB 00:00 2024-05-27T12:32:35.823 INFO:teuthology.orchestra.run.smithi097.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 1.5 MB/s | 1.8 MB 00:01 2024-05-27T12:32:35.831 INFO:teuthology.orchestra.run.smithi094.stdout:(122/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-05-27T12:32:35.864 INFO:teuthology.orchestra.run.smithi094.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-27T12:32:35.948 INFO:teuthology.orchestra.run.smithi094.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-05-27T12:32:35.982 INFO:teuthology.orchestra.run.smithi094.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-05-27T12:32:36.061 INFO:teuthology.orchestra.run.smithi094.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 90 kB/s | 18 kB 00:00 2024-05-27T12:32:36.190 INFO:teuthology.orchestra.run.smithi097.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 50 kB/s | 37 kB 00:00 2024-05-27T12:32:36.233 INFO:teuthology.orchestra.run.smithi094.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-05-27T12:32:36.258 INFO:teuthology.orchestra.run.smithi094.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 71 kB/s | 19 kB 00:00 2024-05-27T12:32:36.284 INFO:teuthology.orchestra.run.smithi094.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 89 kB/s | 20 kB 00:00 2024-05-27T12:32:36.351 INFO:teuthology.orchestra.run.smithi094.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 510 kB/s | 46 kB 00:00 2024-05-27T12:32:36.376 INFO:teuthology.orchestra.run.smithi094.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-05-27T12:32:36.426 INFO:teuthology.orchestra.run.smithi094.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-05-27T12:32:36.452 INFO:teuthology.orchestra.run.smithi094.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 578 kB/s | 58 kB 00:00 2024-05-27T12:32:36.465 INFO:teuthology.orchestra.run.smithi097.stdout:(155/159): NetworkManager-tui-1.47.90-1.el9.x86 240 kB/s | 248 kB 00:01 2024-05-27T12:32:36.485 INFO:teuthology.orchestra.run.smithi094.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-27T12:32:36.511 INFO:teuthology.orchestra.run.smithi094.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-05-27T12:32:36.553 INFO:teuthology.orchestra.run.smithi094.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-05-27T12:32:36.578 INFO:teuthology.orchestra.run.smithi094.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-27T12:32:36.603 INFO:teuthology.orchestra.run.smithi094.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-05-27T12:32:36.637 INFO:teuthology.orchestra.run.smithi094.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-27T12:32:36.662 INFO:teuthology.orchestra.run.smithi094.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-27T12:32:36.687 INFO:teuthology.orchestra.run.smithi094.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 432 kB/s | 36 kB 00:00 2024-05-27T12:32:36.713 INFO:teuthology.orchestra.run.smithi094.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-27T12:32:36.747 INFO:teuthology.orchestra.run.smithi094.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-27T12:32:36.765 INFO:teuthology.orchestra.run.smithi097.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 220 kB/s | 207 kB 00:00 2024-05-27T12:32:36.780 INFO:teuthology.orchestra.run.smithi094.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 977 kB/s | 90 kB 00:00 2024-05-27T12:32:36.806 INFO:teuthology.orchestra.run.smithi094.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 344 kB/s | 20 kB 00:00 2024-05-27T12:32:36.839 INFO:teuthology.orchestra.run.smithi094.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T12:32:36.873 INFO:teuthology.orchestra.run.smithi094.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-27T12:32:36.998 INFO:teuthology.orchestra.run.smithi094.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-27T12:32:37.166 INFO:teuthology.orchestra.run.smithi097.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 266 kB/s | 260 kB 00:00 2024-05-27T12:32:37.207 INFO:teuthology.orchestra.run.smithi094.stdout:(149/159): librados2-19.0.0-3154.g81797d87.el9. 9.2 MB/s | 3.4 MB 00:00 2024-05-27T12:32:37.324 INFO:teuthology.orchestra.run.smithi094.stdout:(150/159): librbd1-19.0.0-3154.g81797d87.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-27T12:32:37.666 INFO:teuthology.orchestra.run.smithi097.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-27T12:32:38.091 INFO:teuthology.orchestra.run.smithi097.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-05-27T12:32:38.106 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-05-27T12:32:38.106 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 226 MB 00:21 2024-05-27T12:32:38.191 INFO:teuthology.orchestra.run.smithi094.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 52 kB/s | 41 kB 00:00 2024-05-27T12:32:38.466 INFO:teuthology.orchestra.run.smithi094.stdout:(152/159): NetworkManager-1.47.90-1.el9.x86_64. 1.6 MB/s | 2.3 MB 00:01 2024-05-27T12:32:38.625 INFO:teuthology.orchestra.run.smithi094.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 1.3 MB/s | 1.8 MB 00:01 2024-05-27T12:32:39.142 INFO:teuthology.orchestra.run.smithi094.stdout:(154/159): NetworkManager-tui-1.47.90-1.el9.x86 261 kB/s | 248 kB 00:00 2024-05-27T12:32:39.326 INFO:teuthology.orchestra.run.smithi094.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 46 kB/s | 37 kB 00:00 2024-05-27T12:32:39.659 INFO:teuthology.orchestra.run.smithi094.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 201 kB/s | 207 kB 00:01 2024-05-27T12:32:40.151 INFO:teuthology.orchestra.run.smithi094.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 258 kB/s | 260 kB 00:01 2024-05-27T12:32:40.460 INFO:teuthology.orchestra.run.smithi094.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-27T12:32:40.827 INFO:teuthology.orchestra.run.smithi094.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-05-27T12:32:40.834 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-05-27T12:32:40.834 INFO:teuthology.orchestra.run.smithi094.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-27T12:32:41.355 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-05-27T12:32:41.612 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-05-27T12:32:41.613 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-05-27T12:32:44.113 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-05-27T12:32:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-05-27T12:32:44.265 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-05-27T12:32:44.365 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-05-27T12:32:44.365 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-05-27T12:32:47.086 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-05-27T12:32:47.086 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-05-27T12:32:51.231 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-05-27T12:32:51.358 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T12:32:51.437 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T12:32:51.566 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T12:32:51.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T12:32:51.659 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T12:32:51.763 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T12:32:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T12:32:52.281 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:32:52.443 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:32:52.474 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:32:52.607 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:32:52.639 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-27T12:32:52.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T12:32:53.060 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T12:32:53.130 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T12:32:53.154 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:32:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:32:53.347 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-27T12:32:53.433 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-27T12:32:53.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T12:32:53.524 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T12:32:53.804 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-05-27T12:32:53.851 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T12:32:53.912 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T12:32:53.991 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T12:32:54.082 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T12:32:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T12:32:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T12:32:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T12:32:54.278 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T12:32:54.371 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T12:32:54.459 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T12:32:54.658 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T12:32:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:32:54.994 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:32:55.020 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:32:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:32:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-27T12:32:55.273 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T12:32:55.281 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T12:32:55.396 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T12:32:55.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-27T12:32:55.489 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-27T12:32:55.591 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-27T12:32:55.643 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-27T12:32:55.645 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T12:32:55.706 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-27T12:32:55.718 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T12:32:55.742 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:32:55.756 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-27T12:32:55.796 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-27T12:32:55.893 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:32:55.912 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-27T12:32:55.918 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-27T12:32:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-27T12:32:56.003 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-27T12:32:56.036 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T12:32:56.041 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-27T12:32:56.094 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T12:32:56.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-27T12:32:56.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-27T12:32:56.285 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 34/170 2024-05-27T12:32:56.336 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 35/170 2024-05-27T12:32:56.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 36/170 2024-05-27T12:32:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T12:32:56.485 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-27T12:32:56.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-27T12:32:56.676 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-27T12:32:56.774 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T12:32:56.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-27T12:32:56.924 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-27T12:32:56.984 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T12:32:57.072 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T12:32:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:57.353 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:57.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-27T12:32:57.670 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T12:32:57.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-27T12:32:57.766 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-27T12:32:57.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-27T12:32:57.861 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T12:32:57.873 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-27T12:32:57.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 47/170 2024-05-27T12:32:57.979 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T12:32:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-27T12:32:58.052 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-27T12:32:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-27T12:32:58.252 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-27T12:32:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-27T12:32:58.320 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-27T12:32:58.364 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-27T12:32:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-27T12:32:58.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-27T12:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T12:32:58.545 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T12:32:58.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-27T12:32:58.584 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T12:32:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-27T12:32:58.651 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T12:32:58.692 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T12:32:58.754 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T12:32:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-27T12:32:58.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T12:32:58.862 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-27T12:32:58.929 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-27T12:32:59.019 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-27T12:32:59.057 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T12:32:59.111 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T12:32:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T12:32:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-27T12:32:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 34/170 2024-05-27T12:32:59.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T12:32:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 35/170 2024-05-27T12:32:59.343 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T12:32:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 36/170 2024-05-27T12:32:59.389 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T12:32:59.424 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T12:32:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T12:32:59.492 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-27T12:32:59.575 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-27T12:32:59.674 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T12:32:59.696 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-27T12:32:59.874 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T12:32:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-27T12:32:59.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T12:32:59.962 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T12:32:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-27T12:33:00.008 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T12:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T12:33:00.119 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T12:33:00.175 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T12:33:00.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T12:33:00.273 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T12:33:00.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T12:33:00.661 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-27T12:33:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T12:33:00.783 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-27T12:33:00.852 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-27T12:33:00.899 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T12:33:00.918 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-27T12:33:00.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T12:33:01.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T12:33:01.026 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-27T12:33:01.087 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 47/170 2024-05-27T12:33:01.090 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T12:33:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-27T12:33:01.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T12:33:01.324 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T12:33:01.398 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-27T12:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-27T12:33:01.540 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-27T12:33:01.662 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T12:33:01.722 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T12:33:01.762 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T12:33:01.831 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T12:33:01.877 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T12:33:01.940 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T12:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T12:33:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T12:33:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T12:33:02.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T12:33:02.308 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T12:33:02.323 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T12:33:02.357 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T12:33:02.440 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T12:33:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T12:33:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T12:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T12:33:02.549 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T12:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T12:33:02.604 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T12:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T12:33:02.711 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T12:33:02.751 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T12:33:02.865 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T12:33:02.984 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T12:33:03.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T12:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T12:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T12:33:03.128 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T12:33:03.138 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T12:33:03.185 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T12:33:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T12:33:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T12:33:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T12:33:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T12:33:03.431 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T12:33:03.532 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T12:33:03.582 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T12:33:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T12:33:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T12:33:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T12:33:03.916 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T12:33:03.963 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T12:33:04.063 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T12:33:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T12:33:04.132 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T12:33:05.080 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T12:33:05.132 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T12:33:05.166 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T12:33:05.295 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T12:33:05.368 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T12:33:05.422 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T12:33:05.533 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T12:33:05.593 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T12:33:05.697 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T12:33:05.796 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T12:33:05.832 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T12:33:05.840 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T12:33:05.868 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T12:33:05.873 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T12:33:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T12:33:06.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T12:33:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-27T12:33:06.414 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-27T12:33:06.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 104/170 2024-05-27T12:33:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T12:33:06.857 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T12:33:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T12:33:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T12:33:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-27T12:33:07.195 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-27T12:33:07.221 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 108/170 2024-05-27T12:33:08.643 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T12:33:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T12:33:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T12:33:08.808 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T12:33:09.219 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T12:33:09.257 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-27T12:33:09.377 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-27T12:33:09.413 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 104/170 2024-05-27T12:33:09.478 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T12:33:09.956 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:10.001 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:10.021 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T12:33:10.051 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-27T12:33:10.158 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-27T12:33:10.212 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 108/170 2024-05-27T12:33:10.648 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:10.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-27T12:33:10.858 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-27T12:33:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T12:33:10.858 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T12:33:10.858 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:10.906 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-27T12:33:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:13.564 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-27T12:33:13.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-27T12:33:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-27T12:33:13.776 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T12:33:13.776 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T12:33:13.776 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:13.820 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-05-27T12:33:24.038 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-05-27T12:33:24.039 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-05-27T12:33:24.039 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-05-27T12:33:24.039 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:24.273 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-27T12:33:24.320 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-27T12:33:24.320 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:24.320 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T12:33:24.320 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:25.940 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-27T12:33:26.025 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-27T12:33:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T12:33:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T12:33:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T12:33:26.027 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:26.282 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-27T12:33:26.362 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-27T12:33:26.363 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:26.363 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T12:33:26.363 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T12:33:26.363 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T12:33:26.363 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:26.849 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-27T12:33:26.929 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-27T12:33:26.930 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:26.930 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T12:33:26.930 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T12:33:26.930 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T12:33:26.931 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-27T12:33:27.032 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-05-27T12:33:27.032 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-05-27T12:33:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:27.058 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T12:33:27.118 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T12:33:27.185 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T12:33:27.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-27T12:33:27.308 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-27T12:33:27.308 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:27.309 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T12:33:27.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:27.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T12:33:27.498 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T12:33:27.580 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T12:33:27.680 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T12:33:27.850 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T12:33:27.912 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T12:33:27.953 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T12:33:28.010 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T12:33:28.686 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-27T12:33:28.726 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-27T12:33:28.766 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 128/170 2024-05-27T12:33:28.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 129/170 2024-05-27T12:33:28.902 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-27T12:33:28.930 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-27T12:33:29.006 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-27T12:33:29.007 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:29.007 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T12:33:29.007 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T12:33:29.007 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T12:33:29.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:29.253 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T12:33:29.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:29.821 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T12:33:29.890 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T12:33:30.084 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T12:33:30.157 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T12:33:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-27T12:33:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-27T12:33:30.398 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T12:33:30.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T12:33:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T12:33:30.653 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T12:33:30.830 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T12:33:30.908 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T12:33:30.949 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T12:33:30.981 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T12:33:31.358 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-27T12:33:31.396 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-27T12:33:31.543 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-27T12:33:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-27T12:33:31.664 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-27T12:33:31.723 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 128/170 2024-05-27T12:33:31.861 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 129/170 2024-05-27T12:33:31.897 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-27T12:33:31.970 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 133/170 2024-05-27T12:33:32.006 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-27T12:33:32.075 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-27T12:33:32.075 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:32.075 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T12:33:32.075 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T12:33:32.075 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T12:33:32.076 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:32.140 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-27T12:33:32.178 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-27T12:33:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 136/170 2024-05-27T12:33:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 137/170 2024-05-27T12:33:33.221 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-27T12:33:33.266 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-27T12:33:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T12:33:33.426 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-27T12:33:33.495 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-27T12:33:33.496 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:33.496 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T12:33:33.496 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T12:33:33.496 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T12:33:33.496 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:33.543 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-27T12:33:33.616 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-27T12:33:33.616 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:33.616 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T12:33:33.616 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:33.686 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T12:33:33.757 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:34.120 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T12:33:34.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:34.370 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-27T12:33:34.400 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-27T12:33:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-27T12:33:34.732 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 133/170 2024-05-27T12:33:34.760 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-27T12:33:34.839 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-27T12:33:34.839 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:34.840 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T12:33:34.840 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T12:33:34.840 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T12:33:34.840 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:34.894 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-27T12:33:34.933 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-27T12:33:35.046 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 136/170 2024-05-27T12:33:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 137/170 2024-05-27T12:33:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T12:33:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T12:33:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-27T12:33:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-27T12:33:36.342 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:36.342 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T12:33:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-27T12:33:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-27T12:33:36.494 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:36.494 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T12:33:36.494 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T12:33:36.494 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T12:33:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:36.865 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T12:33:36.942 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:39.214 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 143/170 2024-05-27T12:33:39.297 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 144/170 2024-05-27T12:33:39.362 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 145/170 2024-05-27T12:33:39.398 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T12:33:39.435 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T12:33:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 148/170 2024-05-27T12:33:39.770 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T12:33:39.800 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 150/170 2024-05-27T12:33:39.840 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T12:33:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T12:33:39.907 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T12:33:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:41.996 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 143/170 2024-05-27T12:33:42.041 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 144/170 2024-05-27T12:33:42.095 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 145/170 2024-05-27T12:33:42.129 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T12:33:42.166 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T12:33:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 155/170 2024-05-27T12:33:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T12:33:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T12:33:42.432 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 148/170 2024-05-27T12:33:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T12:33:42.532 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T12:33:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T12:33:42.569 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 150/170 2024-05-27T12:33:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T12:33:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T12:33:42.616 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T12:33:42.665 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T12:33:42.678 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T12:33:42.678 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T12:33:42.718 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T12:33:42.794 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T12:33:42.844 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T12:33:42.845 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T12:33:42.989 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T12:33:43.030 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T12:33:43.061 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:43.408 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:43.408 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T12:33:43.577 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T12:33:43.623 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T12:33:43.635 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:43.681 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:43.735 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T12:33:43.741 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T12:33:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T12:33:43.993 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T12:33:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:45.104 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 155/170 2024-05-27T12:33:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T12:33:45.293 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T12:33:45.440 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T12:33:45.473 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T12:33:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T12:33:45.542 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T12:33:45.581 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T12:33:45.581 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T12:33:45.691 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T12:33:45.738 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T12:33:45.738 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T12:33:45.883 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T12:33:45.922 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T12:33:45.954 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:45.955 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T12:33:46.247 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T12:33:46.363 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T12:33:46.408 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T12:33:46.527 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T12:33:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T12:33:46.796 INFO:teuthology.orchestra.run.smithi094.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T12:33:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:49.833 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 1/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 3/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 4/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g81797 5/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 6/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 7/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 11/170 2024-05-27T12:33:49.834 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 12/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 13/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 15/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 16/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 17/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 18/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 19/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 20/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 21/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 22/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 23/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 24/170 2024-05-27T12:33:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 25/170 2024-05-27T12:33:49.836 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 26/170 2024-05-27T12:33:49.836 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 27/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 28/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 29/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 30/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 31/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 32/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 33/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 34/170 2024-05-27T12:33:49.848 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 35/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 36/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 37/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 38/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 39/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T12:33:49.849 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T12:33:49.850 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T12:33:49.851 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T12:33:49.852 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T12:33:49.853 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T12:33:49.854 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T12:33:49.855 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T12:33:49.856 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 149/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 151/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T12:33:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T12:33:49.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T12:33:52.172 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:52.173 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.174 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T12:33:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T12:33:52.177 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T12:33:52.178 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T12:33:52.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T12:33:52.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T12:33:52.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T12:33:52.183 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T12:33:52.184 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T12:33:52.185 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T12:33:52.185 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T12:33:52.185 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T12:33:52.185 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:33:52.185 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-05-27T12:33:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 1/170 2024-05-27T12:33:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 3/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 4/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g81797 5/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 6/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 7/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 11/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 12/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 13/170 2024-05-27T12:33:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 15/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 16/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 17/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 18/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 19/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 20/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 21/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 22/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 23/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 24/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 25/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 26/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 27/170 2024-05-27T12:33:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 28/170 2024-05-27T12:33:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 29/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 30/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 31/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 32/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 33/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 34/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 35/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 36/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 37/170 2024-05-27T12:33:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 38/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 39/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T12:33:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T12:33:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T12:33:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T12:33:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T12:33:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T12:33:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T12:33:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T12:33:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 149/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 151/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T12:33:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T12:33:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T12:33:52.873 DEBUG:teuthology.parallel:result is None 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T12:33:54.772 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T12:33:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.774 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T12:33:54.775 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.776 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T12:33:54.777 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T12:33:54.778 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T12:33:54.779 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T12:33:54.780 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T12:33:54.781 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T12:33:54.782 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T12:33:54.783 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T12:33:54.784 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:33:54.785 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-05-27T12:33:55.452 DEBUG:teuthology.parallel:result is None 2024-05-27T12:33:55.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-27T12:33:55.675 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T12:33:55.730 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3154.g81797d87.el9 2024-05-27T12:33:55.730 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g81797d87.el9 2024-05-27T12:33:55.731 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g81797d87 is installed. 2024-05-27T12:33:55.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-27T12:33:55.950 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T12:33:56.002 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3154.g81797d87.el9 2024-05-27T12:33:56.003 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g81797d87.el9 2024-05-27T12:33:56.003 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g81797d87 is installed. 2024-05-27T12:33:56.004 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T12:33:56.005 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:56.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T12:33:56.076 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:56.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T12:33:56.141 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T12:33:56.142 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:56.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T12:33:56.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T12:33:56.315 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:56.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T12:33:56.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T12:33:56.484 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T12:33:56.484 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:56.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T12:33:56.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T12:33:56.658 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:56.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T12:33:56.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T12:33:56.843 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T12:33:56.843 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:56.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T12:33:56.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T12:33:57.018 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:57.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T12:33:57.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T12:33:57.231 INFO:teuthology.run_tasks:Running task ceph... 2024-05-27T12:33:57.312 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-27T12:33:57.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-05-27T12:33:57.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-05-27T12:33:57.405 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-27T12:33:57.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T12:33:57.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T12:33:57.491 INFO:tasks.ceph:Creating extra log directories... 2024-05-27T12:33:57.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T12:33:57.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T12:33:57.588 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-27T12:33:57.588 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-27T12:33:57.588 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-27T12:33:57.590 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:57.590 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T12:33:57.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T12:33:57.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T12:33:57.878 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:57.878 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T12:33:57.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T12:33:58.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T12:33:58.273 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-27T12:33:58.273 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-27T12:33:58.273 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7727886', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7727886', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-27_12:15:13-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-clone-stats', 'sha1': '762bfab4bbf713ca5c79093d382422db341b04da'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 791, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-clone-stats', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_762bfab4bbf713ca5c79093d382422db341b04da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '762bfab4bbf713ca5c79093d382422db341b04da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFxzH66mEcX5t06aaIW/gkpvBw/ovdyjzzw6ljDcUZZat7XFtuABWNAwi6Am6NP/ngDYsLl5eKxgU4NwE4Q3qhc=', 'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNOvNGkFAta5F0vke0nI+YnFLVGMO2EgTML2+cceEjfLRz55tefDqZ7r+U7BEelA7dCEl6/C8DpNkf8aPYol0U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e7ae7580c417d6ab27fda98fecadca5d9f85d63e', 'timestamp': '2024-05-27_12:15:13', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-27T12:33:58.274 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T12:33:58.278 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T12:33:58.335 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T12:33:58.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T12:33:58.447 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:33:58.448 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T12:33:58.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T12:33:58.498 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-05-27T12:33:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-05-27 12:33:47.198030811 +0000 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-05-27 12:30:55.801574915 +0000 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-05-27 12:30:55.801574915 +0000 2024-05-27T12:33:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-05-27 12:30:55.801574915 +0000 2024-05-27T12:33:58.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T12:33:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-05-27T12:33:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-05-27T12:33:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000344856 s, 1.5 MB/s 2024-05-27T12:33:58.669 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T12:33:58.709 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-05-27T12:33:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T12:33:58.790 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-05-27 12:33:47.199030794 +0000 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-05-27 12:30:56.360565606 +0000 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-05-27 12:30:56.360565606 +0000 2024-05-27T12:33:58.791 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-05-27 12:30:56.360565606 +0000 2024-05-27T12:33:58.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T12:33:58.896 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-05-27T12:33:58.896 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-05-27T12:33:58.896 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00019243 s, 2.7 MB/s 2024-05-27T12:33:58.899 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T12:33:58.941 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-05-27T12:33:59.030 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T12:33:59.030 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:59.030 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-27T12:33:59.030 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:59.030 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:59.031 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-05-27 12:33:47.200030778 +0000 2024-05-27T12:33:59.031 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-05-27 12:30:56.922556248 +0000 2024-05-27T12:33:59.031 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-05-27 12:30:56.922556248 +0000 2024-05-27T12:33:59.031 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-05-27 12:30:56.922556248 +0000 2024-05-27T12:33:59.031 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T12:33:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-05-27T12:33:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-05-27T12:33:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000237278 s, 2.2 MB/s 2024-05-27T12:33:59.097 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T12:33:59.143 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-05-27T12:33:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-05-27 12:33:47.201030762 +0000 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-05-27 12:30:57.472547089 +0000 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-05-27 12:30:57.472547089 +0000 2024-05-27T12:33:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-05-27 12:30:57.472547089 +0000 2024-05-27T12:33:59.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T12:33:59.297 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-05-27T12:33:59.297 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-05-27T12:33:59.298 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000286967 s, 1.8 MB/s 2024-05-27T12:33:59.301 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T12:33:59.351 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T12:33:59.351 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:33:59.351 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T12:33:59.409 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T12:33:59.409 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-05-27T12:33:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T12:33:59.463 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:59.463 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-27T12:33:59.463 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:59.463 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:59.464 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-05-27 12:33:44.455066826 +0000 2024-05-27T12:33:59.464 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-05-27 12:30:58.181589807 +0000 2024-05-27T12:33:59.464 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-05-27 12:30:58.181589807 +0000 2024-05-27T12:33:59.464 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-05-27 12:30:58.181589807 +0000 2024-05-27T12:33:59.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T12:33:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-05-27T12:33:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-05-27T12:33:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000351032 s, 1.5 MB/s 2024-05-27T12:33:59.547 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T12:33:59.602 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-05-27T12:33:59.654 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T12:33:59.654 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:59.654 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-27T12:33:59.654 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:59.654 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:59.655 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-05-27 12:33:44.456066809 +0000 2024-05-27T12:33:59.655 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-05-27 12:30:58.775579919 +0000 2024-05-27T12:33:59.655 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-05-27 12:30:58.775579919 +0000 2024-05-27T12:33:59.655 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-05-27 12:30:58.775579919 +0000 2024-05-27T12:33:59.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T12:33:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-05-27T12:33:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-05-27T12:33:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000437906 s, 1.2 MB/s 2024-05-27T12:33:59.736 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T12:33:59.789 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-05-27T12:33:59.838 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-05-27 12:33:44.456066809 +0000 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-05-27 12:30:59.364570114 +0000 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-05-27 12:30:59.364570114 +0000 2024-05-27T12:33:59.839 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-05-27 12:30:59.364570114 +0000 2024-05-27T12:33:59.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T12:33:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-05-27T12:33:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-05-27T12:33:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000378272 s, 1.4 MB/s 2024-05-27T12:33:59.923 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T12:33:59.977 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-05-27T12:34:00.027 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T12:34:00.027 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T12:34:00.027 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-05-27 12:33:44.457066793 +0000 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-05-27 12:30:59.915560942 +0000 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-05-27 12:30:59.915560942 +0000 2024-05-27T12:34:00.028 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-05-27 12:30:59.915560942 +0000 2024-05-27T12:34:00.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T12:34:00.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-05-27T12:34:00.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-05-27T12:34:00.103 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000383953 s, 1.3 MB/s 2024-05-27T12:34:00.108 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T12:34:00.163 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:00.164 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T12:34:00.164 INFO:tasks.ceph:Generating config... 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] debug client = 20 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] debug ms = 1 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] pid file = 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-27T12:34:00.165 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-27T12:34:00.165 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-27T12:34:00.166 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-27T12:34:00.167 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-27T12:34:00.167 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-27T12:34:00.168 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-27T12:34:00.168 INFO:tasks.ceph:Setting up mon.a... 2024-05-27T12:34:00.169 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-27T12:34:00.298 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-05-27T12:34:00.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-27T12:34:00.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T12:34:00.505 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.b', '172.21.15.97'), ('mon.c', '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]')] 2024-05-27T12:34:00.505 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-27T12:34:00.506 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:00.506 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T12:34:00.560 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --add b 172.21.15.97 --addv c '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:00.712 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:00.712 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid b6a8d677-9e12-4245-aa4c-e7c6ae09d474 2024-05-27T12:34:00.712 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:fsid b6a8d677-9e12-4245-aa4c-e7c6ae09d474 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-05-27T12:34:00.714402+0000 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-05-27T12:34:00.714402+0000 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] mon.c 2024-05-27T12:34:00.713 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-27T12:34:00.716 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T12:34:00.765 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b6a8d677-9e12-4245-aa4c-e7c6ae09d474... 2024-05-27T12:34:00.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T12:34:00.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T12:34:00.899 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-05-27T12:34:00.899 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.900 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-05-27T12:34:00.901 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-05-27T12:34:00.902 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = b6a8d677-9e12-4245-aa4c-e7c6ae09d474 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.903 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-05-27T12:34:00.904 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-05-27T12:34:00.905 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-05-27T12:34:00.906 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug finisher = 20 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-05-27T12:34:00.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-05-27T12:34:00.908 INFO:teuthology.orchestra.run.smithi094.stdout: mon op complaint time = 120 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 600 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-05-27T12:34:00.909 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds = 20 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: debug mds balancer = 20 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug frag = True 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: mds debug scatterstat = True 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: mds op complaint time = 180 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: mds verify scatter = True 2024-05-27T12:34:00.910 INFO:teuthology.orchestra.run.smithi094.stdout: osd op complaint time = 180 2024-05-27T12:34:00.911 INFO:teuthology.orchestra.run.smithi094.stdout: rados mon op timeout = 900 2024-05-27T12:34:00.911 INFO:teuthology.orchestra.run.smithi094.stdout: rados osd op timeout = 900 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-05-27T12:34:00.917 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.94,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on skipped message = False 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = b6a8d677-9e12-4245-aa4c-e7c6ae09d474 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-05-27T12:34:00.918 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.919 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-05-27T12:34:00.920 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-05-27T12:34:00.921 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug finisher = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-05-27T12:34:00.922 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T12:34:00.923 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: mon down mkfs grace = 300 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2024-05-27T12:34:00.924 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2024-05-27T12:34:00.925 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-05-27T12:34:00.926 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-05-27T12:34:00.948 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-27T12:34:00.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-27T12:34:01.080 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-27T12:34:01.080 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:01.080 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-27T12:34:01.129 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:01.129 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-27T12:34:01.181 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:34:01.181 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:01.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T12:34:01.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T12:34:01.307 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:01.308 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:01.405 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:34:01.406 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:01.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T12:34:01.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T12:34:01.543 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:01.544 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:01.647 INFO:tasks.ceph:Setting up mon nodes... 2024-05-27T12:34:01.647 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-27T12:34:01.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T12:34:01.802 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T12:34:01.810 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T12:34:01.999 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T12:34:02.009 INFO:tasks.ceph:Setting up mds nodes... 2024-05-27T12:34:02.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T12:34:02.160 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T12:34:02.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-27T12:34:02.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T12:34:02.400 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T12:34:02.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-27T12:34:02.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T12:34:02.636 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T12:34:02.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-27T12:34:02.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T12:34:02.894 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T12:34:02.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-27T12:34:02.987 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-27T12:34:02.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T12:34:03.110 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-27T12:34:03.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-27T12:34:03.278 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-27T12:34:03.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-27T12:34:03.435 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-27T12:34:03.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-27T12:34:03.598 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-27T12:34:03.642 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-27T12:34:03.642 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T12:34:03.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-27T12:34:03.733 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:03.733 INFO:tasks.ceph:role: osd.0 2024-05-27T12:34:03.733 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:34:03.733 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T12:34:03.813 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:03.814 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:03.858 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-05-27T12:34:03.863 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-05-27T12:34:03.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-27T12:34:03.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-27T12:34:04.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-27T12:34:04.117 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:04.117 INFO:tasks.ceph:role: osd.1 2024-05-27T12:34:04.118 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:34:04.118 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T12:34:04.191 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:04.192 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:04.193 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:04.193 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:04.193 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:04.237 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-05-27T12:34:04.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-05-27T12:34:04.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-27T12:34:04.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-27T12:34:04.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-27T12:34:04.494 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:04.494 INFO:tasks.ceph:role: osd.2 2024-05-27T12:34:04.494 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:34:04.494 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:04.617 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-05-27T12:34:04.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-05-27T12:34:04.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-27T12:34:04.720 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-27T12:34:04.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-27T12:34:04.858 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:04.858 INFO:tasks.ceph:role: osd.3 2024-05-27T12:34:04.859 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:34:04.859 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T12:34:04.934 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:04.935 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:04.976 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-05-27T12:34:04.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-05-27T12:34:04.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-27T12:34:05.084 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-27T12:34:05.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.283+0000 7f9e6beb1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-27T12:34:05.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.283+0000 7f9e6beb1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-27T12:34:05.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.283+0000 7f9e6beb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T12:34:05.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.283+0000 7f9e6beb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T12:34:05.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.283+0000 7f9e6beb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T12:34:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:05.287+0000 7f9e6beb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-27T12:34:08.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T12:34:08.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:08.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.453+0000 7fc14dbdc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-27T12:34:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.454+0000 7fc14dbdc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-27T12:34:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.454+0000 7fc14dbdc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T12:34:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.454+0000 7fc14dbdc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T12:34:08.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.454+0000 7fc14dbdc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T12:34:08.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:08.456+0000 7fc14dbdc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-27T12:34:11.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T12:34:11.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.903+0000 7f6cb345d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-27T12:34:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.903+0000 7f6cb345d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-27T12:34:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.903+0000 7f6cb345d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T12:34:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.903+0000 7f6cb345d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T12:34:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.903+0000 7f6cb345d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T12:34:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:11.905+0000 7f6cb345d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-27T12:34:14.868 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T12:34:14.944 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.064+0000 7f16855c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-27T12:34:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.064+0000 7f16855c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-27T12:34:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.064+0000 7f16855c4640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T12:34:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.064+0000 7f16855c4640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T12:34:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.064+0000 7f16855c4640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T12:34:15.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:15.067+0000 7f16855c4640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-27T12:34:18.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T12:34:18.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-27T12:34:18.200 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:18.200 INFO:tasks.ceph:role: osd.4 2024-05-27T12:34:18.200 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:34:18.200 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T12:34:18.290 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:18.290 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:18.291 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:18.703 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-05-27T12:34:18.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-05-27T12:34:18.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-27T12:34:18.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-27T12:34:18.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-27T12:34:19.028 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:19.029 INFO:tasks.ceph:role: osd.5 2024-05-27T12:34:19.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:34:19.029 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:19.107 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:19.108 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:19.108 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:19.461 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-05-27T12:34:19.466 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-05-27T12:34:19.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-27T12:34:19.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-27T12:34:19.637 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-27T12:34:19.702 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:19.703 INFO:tasks.ceph:role: osd.6 2024-05-27T12:34:19.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:34:19.703 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:19.778 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:19.779 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:19.779 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:19.779 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:20.163 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-05-27T12:34:20.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-05-27T12:34:20.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-27T12:34:20.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-27T12:34:20.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-27T12:34:20.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T12:34:20.428 INFO:tasks.ceph:role: osd.7 2024-05-27T12:34:20.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:34:20.429 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T12:34:20.509 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T12:34:20.510 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T12:34:20.911 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-05-27T12:34:20.919 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-05-27T12:34:20.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-27T12:34:21.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-27T12:34:21.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.323+0000 7fd51b011640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-27T12:34:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.323+0000 7fd51b011640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-27T12:34:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.323+0000 7fd51b011640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T12:34:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.323+0000 7fd51b011640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T12:34:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.323+0000 7fd51b011640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T12:34:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:21.327+0000 7fd51b011640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-27T12:34:24.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T12:34:24.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:24.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.505+0000 7fcf046b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-27T12:34:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.506+0000 7fcf046b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-27T12:34:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.506+0000 7fcf046b5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T12:34:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.506+0000 7fcf046b5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T12:34:24.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.506+0000 7fcf046b5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T12:34:24.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:24.508+0000 7fcf046b5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-27T12:34:27.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T12:34:27.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:27.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.937+0000 7f41484e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-27T12:34:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.937+0000 7f41484e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-27T12:34:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.937+0000 7f41484e3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T12:34:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.937+0000 7f41484e3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T12:34:27.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.937+0000 7f41484e3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T12:34:27.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:27.940+0000 7f41484e3640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-27T12:34:28.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:34:28.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:34:31.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-27T12:34:31.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.367+0000 7ff2da619640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-27T12:34:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.368+0000 7ff2da619640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-27T12:34:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.368+0000 7ff2da619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T12:34:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.368+0000 7ff2da619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T12:34:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.368+0000 7ff2da619640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T12:34:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:31.370+0000 7ff2da619640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-27T12:34:34.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-27T12:34:34.404 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-27T12:34:34.404 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-27T12:34:34.497 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-27T12:34:34.577 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-27T12:34:34.662 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-27T12:34:34.749 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.749 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-27T12:34:34.835 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-27T12:34:34.919 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:34.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-27T12:34:34.999 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:34.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-27T12:34:35.086 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-27T12:34:35.167 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-27T12:34:35.292 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-27T12:34:35.369 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-27T12:34:35.449 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-27T12:34:35.576 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-27T12:34:35.657 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:35.657 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-27T12:34:35.715 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:35.715 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-27T12:34:35.769 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.769 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-27T12:34:35.827 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:35.827 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-27T12:34:35.880 INFO:tasks.ceph:Adding keys to all mons... 2024-05-27T12:34:35.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T12:34:35.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T12:34:35.965 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC5fVRm9xPrLxAAlqZUU5Xv2T5xYmQ/qfm8VA== 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC6fVRmBSGxCRAAlqoZ3k3248qaaawrgczaKA== 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC6fVRmk7H7FxAA5tzg/wo5AlIjKmlnw5Ntcw== 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC9fVRmhwr0EBAAiVA4TQAPBG2hcQ5rvnDwrQ== 2024-05-27T12:34:35.966 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDAfVRmkEwZGxAA2GvYAQJ2jCQnyb734JBYsg== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDDfVRmC2TlNRAATX3pQr5v3nozFEht3W3ktw== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDHfVRm/qHnAxAAFRu8l+2+uB+N1rH3D35Rpw== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC6fVRmT8gKABAAWwijw+gHiXip0tT2sfaRBg== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC6fVRm4SUSJhAAx+eNO2M0sRsYdhFIUMnnsA== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC6fVRmrrRsNRAAy5Q0sgtxD/1HDx82gqG/sA== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDNfVRmiKNPExAAm9SFMTwayddMameEe0itjg== 2024-05-27T12:34:35.967 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQfVRmbBYtHhAASndxPHopciB2VKHisOwsuw== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDTfVRmD3PsNxAAaBZs2zwb3XH9djFk+J/ahg== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDXfVRmjhL4FRAAhz9w3S1rJYBu9YZCoQpH6A== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC7fVRm82KsBhAApDuhvkqez8bDlFwJfJwTsw== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC7fVRmF/O5EBAA9/DVyfKTzzmOwAtFRf/1og== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC7fVRma9UOGhAAH6lIxB/+y1s/OV0Rq/UxBw== 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2024-05-27T12:34:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC7fVRm6/nHIxAAUkcRRY+5F3coqLK9hPiX/A== 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC5fVRm9xPrLxAAlqZUU5Xv2T5xYmQ/qfm8VA== 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC6fVRmBSGxCRAAlqoZ3k3248qaaawrgczaKA== 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC6fVRmk7H7FxAA5tzg/wo5AlIjKmlnw5Ntcw== 2024-05-27T12:34:35.969 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC9fVRmhwr0EBAAiVA4TQAPBG2hcQ5rvnDwrQ== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDAfVRmkEwZGxAA2GvYAQJ2jCQnyb734JBYsg== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDDfVRmC2TlNRAATX3pQr5v3nozFEht3W3ktw== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDHfVRm/qHnAxAAFRu8l+2+uB+N1rH3D35Rpw== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC6fVRmT8gKABAAWwijw+gHiXip0tT2sfaRBg== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC6fVRm4SUSJhAAx+eNO2M0sRsYdhFIUMnnsA== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC6fVRmrrRsNRAAy5Q0sgtxD/1HDx82gqG/sA== 2024-05-27T12:34:35.970 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDNfVRmiKNPExAAm9SFMTwayddMameEe0itjg== 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDQfVRmbBYtHhAASndxPHopciB2VKHisOwsuw== 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDTfVRmD3PsNxAAaBZs2zwb3XH9djFk+J/ahg== 2024-05-27T12:34:35.971 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDXfVRmjhL4FRAAhz9w3S1rJYBu9YZCoQpH6A== 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7fVRm82KsBhAApDuhvkqez8bDlFwJfJwTsw== 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7fVRmF/O5EBAA9/DVyfKTzzmOwAtFRf/1og== 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7fVRma9UOGhAAH6lIxB/+y1s/OV0Rq/UxBw== 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-05-27T12:34:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7fVRm6/nHIxAAUkcRRY+5F3coqLK9hPiX/A== 2024-05-27T12:34:35.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T12:34:36.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T12:34:36.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:36.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:36.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:36.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:36.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:36.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T12:34:36.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T12:34:37.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:37.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:37.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:37.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T12:34:37.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T12:34:37.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:37.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T12:34:38.469 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-27T12:34:38.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-27T12:34:38.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T12:34:38.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-27T12:34:38.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-27T12:34:38.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T12:34:38.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-27T12:34:39.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-27T12:34:39.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T12:34:39.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-27T12:34:39.355 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:39.363 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T12:34:39.412 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-27T12:34:39.413 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-27T12:34:39.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-27T12:34:39.420 INFO:tasks.ceph.mon.a:Started 2024-05-27T12:34:39.420 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-27T12:34:39.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-27T12:34:39.425 INFO:tasks.ceph.mon.b:Started 2024-05-27T12:34:39.425 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-27T12:34:39.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-27T12:34:39.429 INFO:tasks.ceph.mon.c:Started 2024-05-27T12:34:39.429 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-27T12:34:39.429 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-27T12:34:39.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-27T12:34:39.436 INFO:tasks.ceph.mgr.y:Started 2024-05-27T12:34:39.436 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-27T12:34:39.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-27T12:34:39.441 INFO:tasks.ceph.mgr.x:Started 2024-05-27T12:34:39.441 INFO:tasks.ceph:Setting crush tunables to default 2024-05-27T12:34:39.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-05-27T12:34:39.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.618+0000 7f35fd000640 1 Processor -- start 2024-05-27T12:34:39.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.618+0000 7f35fd000640 1 -- start start 2024-05-27T12:34:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f35f80b8ef0 con 0x7f35f80d2df0 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f35f80b82e0 con 0x7f35f80b7e20 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35fd000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f35f80b85c0 con 0x7f35f815eb70 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.620+0000 7f35f5400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:43342/0 (socket says 172.21.15.94:43342) 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:43036/0 (socket says 172.21.15.94:43036) 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 -- 172.21.15.94:0/2586732747 learned_addr learned my addr 172.21.15.94:0/2586732747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 -- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 msgr2=0x7f35f80d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5400640 1 -- 172.21.15.94:0/2586732747 >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 msgr2=0x7f35f80b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T12:34:39.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 -- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 msgr2=0x7f35f80d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5400640 1 -- 172.21.15.94:0/2586732747 >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 msgr2=0x7f35f80b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 -- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 msgr2=0x7f35f80b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 -- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 msgr2=0x7f35f80b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5e00640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f6800640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T12:34:39.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.621+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5e00640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.94:3300/0 conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f6800640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 -- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 msgr2=0x7f35f80d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 -- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 msgr2=0x7f35f80b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.822+0000 7f35f5400640 1 -- 172.21.15.94:0/2586732747 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f80b88a0 con 0x7f35f80d2df0 2024-05-27T12:34:39.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.824+0000 7f35f5400640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80d2df0 0x7f35f80b78e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35f808dbb0 tx=0x7f35e402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fb9fc45f7e21d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:39.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.825+0000 7f35f4a00640 1 -- 172.21.15.94:0/2586732747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4030070 con 0x7f35f80d2df0 2024-05-27T12:34:39.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.825+0000 7f35f4a00640 1 -- 172.21.15.94:0/2586732747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e4034400 con 0x7f35f80d2df0 2024-05-27T12:34:39.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.825+0000 7f35f4a00640 1 -- 172.21.15.94:0/2586732747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40347f0 con 0x7f35f80d2df0 2024-05-27T12:34:39.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 -- 172.21.15.94:0/2586732747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80d2df0 msgr2=0x7f35f80b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:39.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80d2df0 0x7f35f80b78e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35f808dbb0 tx=0x7f35e402a2c0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 -- 172.21.15.94:0/2586732747 shutdown_connections 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 --2- 172.21.15.94:0/2586732747 >> v2:172.21.15.97:3300/0 conn(0x7f35f80b7e20 0x7f35f80b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80d2df0 0x7f35f80b78e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 --2- 172.21.15.94:0/2586732747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f80d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 -- 172.21.15.94:0/2586732747 >> 172.21.15.94:0/2586732747 conn(0x7f35f8067a50 msgr2=0x7f35f8067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 -- 172.21.15.94:0/2586732747 shutdown_connections 2024-05-27T12:34:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.826+0000 7f35fd000640 1 -- 172.21.15.94:0/2586732747 wait complete. 2024-05-27T12:34:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.829+0000 7f35fd000640 1 Processor -- start 2024-05-27T12:34:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.829+0000 7f35fd000640 1 -- start start 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f814abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f8157c10 0x7f35f8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35f80ba540 con 0x7f35f80b7e20 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35f815ef50 con 0x7f35f8157c10 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35fd000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f35f80ba6c0 con 0x7f35f815eb70 2024-05-27T12:34:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.830+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53844/0 (socket says 172.21.15.94:53844) 2024-05-27T12:34:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 -- 172.21.15.94:0/1465318394 learned_addr learned my addr 172.21.15.94:0/1465318394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5400640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f814abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f6800640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f8157c10 0x7f35f8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 -- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 msgr2=0x7f35f814abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f814abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 -- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f8157c10 msgr2=0x7f35f8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f8157c10 0x7f35f8154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.831+0000 7f35f5e00640 1 -- 172.21.15.94:0/1465318394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f80ce430 con 0x7f35f80b7e20 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35f5e00640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35e80116a0 tx=0x7f35e8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8019070 con 0x7f35f80b7e20 2024-05-27T12:34:39.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e800f9c0 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f80ce710 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e800f040 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.832+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f815f450 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.833+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f35e801c070 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.833+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35e801e790 con 0x7f35f80b7e20 2024-05-27T12:34:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.834+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f8075460 con 0x7f35f80b7e20 2024-05-27T12:34:39.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.837+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f35e800fd90 con 0x7f35f80b7e20 2024-05-27T12:34:39.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.912+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f35f80cfae0 con 0x7f35f80b7e20 2024-05-27T12:34:39.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.915+0000 7f35ee000640 1 -- 172.21.15.94:0/1465318394 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f35e8015e10 con 0x7f35f80b7e20 2024-05-27T12:34:39.914 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 msgr2=0x7f35f81541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35e80116a0 tx=0x7f35e8011b60 comp rx=0 tx=0).stop 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 shutdown_connections 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f8157c10 0x7f35f8154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35f815eb70 0x7f35f814abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 --2- 172.21.15.94:0/1465318394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35f80b7e20 0x7f35f81541d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.916+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 >> 172.21.15.94:0/1465318394 conn(0x7f35f8067a50 msgr2=0x7f35f80d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.917+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 shutdown_connections 2024-05-27T12:34:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:39.917+0000 7f35fd000640 1 -- 172.21.15.94:0/1465318394 wait complete. 2024-05-27T12:34:39.934 INFO:tasks.ceph:check_enable_crimson: False 2024-05-27T12:34:39.935 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-27T12:34:39.935 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:39.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-27T12:34:39.943 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:39.944+0000 7fe699121040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T12:34:39.957 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:39.958+0000 7f021640e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T12:34:40.000 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:40.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-27T12:34:40.110 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:40.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-27T12:34:40.140 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:40.140+0000 7f021640e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T12:34:40.145 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:40.145+0000 7fe699121040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T12:34:40.206 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:40.207+0000 7f021640e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T12:34:40.210 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:34:40.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-27T12:34:40.211 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:40.212+0000 7fe699121040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T12:34:40.326 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:40.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-27T12:34:40.407 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:40.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-27T12:34:40.485 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:40.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-27T12:34:40.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:40.490+0000 7f021640e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T12:34:40.539 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:40.540+0000 7fe699121040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T12:34:40.544 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:40.544+0000 7f021640e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T12:34:40.597 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:40.598+0000 7fe699121040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T12:34:40.608 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:34:40.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-27T12:34:40.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 9e188c11-2c56-451c-8851-a9c63593332f 0 2024-05-27T12:34:40.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.939+0000 7f11cd400640 1 Processor -- start 2024-05-27T12:34:40.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.939+0000 7f11cd400640 1 -- start start 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 0x7f11c8067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11c80ba3f0 0x7f11c80b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c80b66d0 0x7f11c8162480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f11c80b6b40 con 0x7f11c815e900 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f11c80b7840 con 0x7f11c80ba3f0 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cd400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f11c8067f30 con 0x7f11c80b66d0 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11c80ba3f0 0x7f11c80b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11cca00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c80b66d0 0x7f11c8162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11c80ba3f0 0x7f11c80b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49286/0 (socket says 172.21.15.97:49286) 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7400640 1 -- 172.21.15.97:0/790047225 learned_addr learned my addr 172.21.15.97:0/790047225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7e00640 1 --2- 172.21.15.97:0/790047225 >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 0x7f11c8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7400640 1 -- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c80b66d0 msgr2=0x7f11c8162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.941+0000 7f11c7400640 1 --2- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c80b66d0 0x7f11c8162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.942+0000 7f11c7400640 1 -- 172.21.15.97:0/790047225 >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 msgr2=0x7f11c8067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.942+0000 7f11c7400640 1 --2- 172.21.15.97:0/790047225 >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 0x7f11c8067630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.942+0000 7f11c7400640 1 -- 172.21.15.97:0/790047225 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c8068210 con 0x7f11c80ba3f0 2024-05-27T12:34:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.942+0000 7f11c7e00640 1 --2- 172.21.15.97:0/790047225 >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 0x7f11c8067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11c7400640 1 --2- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c80b6190 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11c80d29c0 tx=0x7f11b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19320cb7e5356d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11c6a00640 1 -- 172.21.15.97:0/790047225 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4037070 con 0x7f11c80ba3f0 2024-05-27T12:34:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11c6a00640 1 -- 172.21.15.97:0/790047225 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b4032440 con 0x7f11c80ba3f0 2024-05-27T12:34:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11c6a00640 1 -- 172.21.15.97:0/790047225 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4032810 con 0x7f11c80ba3f0 2024-05-27T12:34:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11cd400640 1 -- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 msgr2=0x7f11c80b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11cd400640 1 --2- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c80b6190 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f11c80d29c0 tx=0x7f11b402a270 comp rx=0 tx=0).stop 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.943+0000 7f11cd400640 1 -- 172.21.15.97:0/790047225 shutdown_connections 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 --2- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c80b66d0 0x7f11c8162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 --2- 172.21.15.97:0/790047225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c80b6190 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 --2- 172.21.15.97:0/790047225 >> v2:172.21.15.94:3300/0 conn(0x7f11c815e900 0x7f11c8067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 -- 172.21.15.97:0/790047225 >> 172.21.15.97:0/790047225 conn(0x7f11c80d1190 msgr2=0x7f11c80d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 -- 172.21.15.97:0/790047225 shutdown_connections 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.944+0000 7f11cd400640 1 -- 172.21.15.97:0/790047225 wait complete. 2024-05-27T12:34:40.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.945+0000 7f11cd400640 1 Processor -- start 2024-05-27T12:34:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.945+0000 7f11cd400640 1 -- start start 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 0x7f11c81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c815e900 0x7f11c80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80d2df0 con 0x7f11c80b66d0 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80d2b10 con 0x7f11c80ba3f0 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.946+0000 7f11cd400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f11c815ece0 con 0x7f11c815e900 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11cca00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c815e900 0x7f11c80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49302/0 (socket says 172.21.15.97:49302) 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 -- 172.21.15.97:0/370543411 learned_addr learned my addr 172.21.15.97:0/370543411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7e00640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 0x7f11c81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 -- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c815e900 msgr2=0x7f11c80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c815e900 0x7f11c80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 -- 172.21.15.97:0/370543411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 msgr2=0x7f11c81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 0x7f11c81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 -- 172.21.15.97:0/370543411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81f7de0 con 0x7f11c80ba3f0 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7e00640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 0x7f11c81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.947+0000 7f11c7400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11b402a2b0 tx=0x7f11b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4037040 con 0x7f11c80ba3f0 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b4042070 con 0x7f11c80ba3f0 2024-05-27T12:34:40.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4030880 con 0x7f11c80ba3f0 2024-05-27T12:34:40.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81f80c0 con 0x7f11c80ba3f0 2024-05-27T12:34:40.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c81f8680 con 0x7f11c80ba3f0 2024-05-27T12:34:40.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11b4030a20 con 0x7f11c80ba3f0 2024-05-27T12:34:40.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.948+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f11b4047b90 con 0x7f11c80ba3f0 2024-05-27T12:34:40.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.949+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c8075460 con 0x7f11c80ba3f0 2024-05-27T12:34:40.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:40.952+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f11b4047d90 con 0x7f11c80ba3f0 2024-05-27T12:34:41.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.030+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9e188c11-2c56-451c-8851-a9c63593332f", "id": 0} v 0) -- 0x7f11c81551b0 con 0x7f11c80ba3f0 2024-05-27T12:34:41.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.037+0000 7f11bfe00640 1 -- 172.21.15.97:0/370543411 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e188c11-2c56-451c-8851-a9c63593332f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f11b4047460 con 0x7f11c80ba3f0 2024-05-27T12:34:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:0 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 msgr2=0x7f11c8161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11b402a2b0 tx=0x7f11b40046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 shutdown_connections 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c80ba3f0 0x7f11c8161910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11c815e900 0x7f11c80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 --2- 172.21.15.97:0/370543411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c80b66d0 0x7f11c81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.038+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 >> 172.21.15.97:0/370543411 conn(0x7f11c80d1190 msgr2=0x7f11c815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.039+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 shutdown_connections 2024-05-27T12:34:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.039+0000 7f11cd400640 1 -- 172.21.15.97:0/370543411 wait complete. 2024-05-27T12:34:41.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new df3b3d8e-1740-4a86-bad2-63ea2e2842e8 1 2024-05-27T12:34:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.223+0000 7f944e600640 1 Processor -- start 2024-05-27T12:34:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.223+0000 7f944e600640 1 -- start start 2024-05-27T12:34:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.224+0000 7f944e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f944e600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 0x7f94480b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f944e600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94480b82e0 con 0x7f94480d2df0 2024-05-27T12:34:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f944e600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f94480b8ef0 con 0x7f94480ba540 2024-05-27T12:34:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f944e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f94480b85c0 con 0x7f944815eb70 2024-05-27T12:34:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 0x7f94480b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:59832/0 (socket says 172.21.15.97:59832) 2024-05-27T12:34:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 -- 172.21.15.97:0/3626212413 learned_addr learned my addr 172.21.15.97:0/3626212413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:41.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 -- 172.21.15.97:0/3626212413 >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 msgr2=0x7f94480b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 --2- 172.21.15.97:0/3626212413 >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 0x7f94480b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 -- 172.21.15.97:0/3626212413 >> v1:172.21.15.97:6789/0 conn(0x7f94480d2df0 legacy=0x7f94480b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.225+0000 7f9447400640 1 -- 172.21.15.97:0/3626212413 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94480b88a0 con 0x7f944815eb70 2024-05-27T12:34:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.226+0000 7f9447e00640 1 --2- 172.21.15.97:0/3626212413 >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 0x7f94480b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.227+0000 7f9447400640 1 --2- 172.21.15.97:0/3626212413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e50 tx=0x7f943802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95d3f784c72d441 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.227+0000 7f9446a00640 1 -- 172.21.15.97:0/3626212413 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438036070 con 0x7f944815eb70 2024-05-27T12:34:41.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.227+0000 7f9446a00640 1 -- 172.21.15.97:0/3626212413 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9438031600 con 0x7f944815eb70 2024-05-27T12:34:41.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.227+0000 7f9446a00640 1 -- 172.21.15.97:0/3626212413 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380319d0 con 0x7f944815eb70 2024-05-27T12:34:41.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 -- 172.21.15.97:0/3626212413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 msgr2=0x7f94480d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 --2- 172.21.15.97:0/3626212413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e50 tx=0x7f943802a7a0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 -- 172.21.15.97:0/3626212413 shutdown_connections 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 --2- 172.21.15.97:0/3626212413 >> v2:172.21.15.97:3300/0 conn(0x7f94480ba540 0x7f94480b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 --2- 172.21.15.97:0/3626212413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f944815eb70 0x7f94480d28b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 -- 172.21.15.97:0/3626212413 >> 172.21.15.97:0/3626212413 conn(0x7f9448067a50 msgr2=0x7f9448067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 -- 172.21.15.97:0/3626212413 shutdown_connections 2024-05-27T12:34:41.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.228+0000 7f944e600640 1 -- 172.21.15.97:0/3626212413 wait complete. 2024-05-27T12:34:41.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.230+0000 7f944e600640 1 Processor -- start 2024-05-27T12:34:41.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.230+0000 7f944e600640 1 -- start start 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.231+0000 7f944e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.231+0000 7f944e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f944815eb70 0x7f9448157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.231+0000 7f944e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 0x7f94481f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f944e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f944815ef50 con 0x7f944815eb70 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f944e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94480b7ff0 con 0x7f94481f4630 2024-05-27T12:34:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f944e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f94480b98c0 con 0x7f94480ba540 2024-05-27T12:34:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:59840/0 (socket says 172.21.15.97:59840) 2024-05-27T12:34:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 0x7f94481f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447400640 1 -- 172.21.15.97:0/2874222327 learned_addr learned my addr 172.21.15.97:0/2874222327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f943ea00640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f944815eb70 0x7f9448157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 -- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 msgr2=0x7f94481476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 -- 172.21.15.97:0/2874222327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f944815eb70 msgr2=0x7f9448157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f944815eb70 0x7f9448157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447e00640 1 -- 172.21.15.97:0/2874222327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94481485e0 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.232+0000 7f9447400640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f9447e00640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 0x7f94481f4aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f944000e880 tx=0x7f944000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440017070 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9440004020 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9448147d40 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400043f0 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9448148300 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9440005990 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.233+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f9440006620 con 0x7f94481f4630 2024-05-27T12:34:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.234+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94480d09c0 con 0x7f94481f4630 2024-05-27T12:34:41.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.237+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f94480d09c0 con 0x7f94481f4630 2024-05-27T12:34:41.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.317+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df3b3d8e-1740-4a86-bad2-63ea2e2842e8", "id": 1} v 0) -- 0x7f9448075460 con 0x7f94481f4630 2024-05-27T12:34:41.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.323+0000 7f943fe00640 1 -- 172.21.15.97:0/2874222327 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df3b3d8e-1740-4a86-bad2-63ea2e2842e8", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f944000c940 con 0x7f94481f4630 2024-05-27T12:34:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-05-27T12:34:41.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 msgr2=0x7f94481f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 0x7f94481f4aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f944000e880 tx=0x7f944000ed40 comp rx=0 tx=0).stop 2024-05-27T12:34:41.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 shutdown_connections 2024-05-27T12:34:41.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94481f4630 0x7f94481f4aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f944815eb70 0x7f9448157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 --2- 172.21.15.97:0/2874222327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f94480ba540 0x7f94481476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 >> 172.21.15.97:0/2874222327 conn(0x7f9448067a50 msgr2=0x7f94480b9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 shutdown_connections 2024-05-27T12:34:41.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.324+0000 7f944e600640 1 -- 172.21.15.97:0/2874222327 wait complete. 2024-05-27T12:34:41.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 90511b5d-5c7a-4221-b94f-5d9053d3f15c 2 2024-05-27T12:34:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.516+0000 7f03fd400640 1 Processor -- start 2024-05-27T12:34:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.516+0000 7f03fd400640 1 -- start start 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03fd400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03fd400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f03f80d3e50 0x7f03f80d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03fd400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f815eee0 con 0x7f03f80d1d70 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03fd400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f03f80d5700 con 0x7f03f80d3e50 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03fd400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80d21a0 con 0x7f03f815eb00 2024-05-27T12:34:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03f6800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03f6800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:59844/0 (socket says 172.21.15.97:59844) 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03f6800640 1 -- 172.21.15.97:0/1263576856 learned_addr learned my addr 172.21.15.97:0/1263576856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.518+0000 7f03f5e00640 1 --2- 172.21.15.97:0/1263576856 >> v2:172.21.15.94:3300/0 conn(0x7f03f80d3e50 0x7f03f80d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.519+0000 7f03f6800640 1 -- 172.21.15.97:0/1263576856 >> v2:172.21.15.94:3300/0 conn(0x7f03f80d3e50 msgr2=0x7f03f80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.519+0000 7f03f6800640 1 --2- 172.21.15.97:0/1263576856 >> v2:172.21.15.94:3300/0 conn(0x7f03f80d3e50 0x7f03f80d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.519+0000 7f03f6800640 1 -- 172.21.15.97:0/1263576856 >> v1:172.21.15.94:6789/0 conn(0x7f03f80d1d70 legacy=0x7f03f80d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.519+0000 7f03f6800640 1 -- 172.21.15.97:0/1263576856 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f80d4660 con 0x7f03f815eb00 2024-05-27T12:34:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03f6800640 1 --2- 172.21.15.97:0/1263576856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009870 tx=0x7f03e802a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb5a1911c2e8abd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03f5400640 1 -- 172.21.15.97:0/1263576856 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8037070 con 0x7f03f815eb00 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03f5400640 1 -- 172.21.15.97:0/1263576856 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e8032440 con 0x7f03f815eb00 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03f5400640 1 -- 172.21.15.97:0/1263576856 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8032810 con 0x7f03f815eb00 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03fd400640 1 -- 172.21.15.97:0/1263576856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 msgr2=0x7f03f80cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.520+0000 7f03fd400640 1 --2- 172.21.15.97:0/1263576856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009870 tx=0x7f03e802a2a0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 -- 172.21.15.97:0/1263576856 shutdown_connections 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 --2- 172.21.15.97:0/1263576856 >> v2:172.21.15.94:3300/0 conn(0x7f03f80d3e50 0x7f03f80d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 --2- 172.21.15.97:0/1263576856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f815eb00 0x7f03f80cfb40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 -- 172.21.15.97:0/1263576856 >> 172.21.15.97:0/1263576856 conn(0x7f03f8067a50 msgr2=0x7f03f8067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 -- 172.21.15.97:0/1263576856 shutdown_connections 2024-05-27T12:34:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.521+0000 7f03fd400640 1 -- 172.21.15.97:0/1263576856 wait complete. 2024-05-27T12:34:41.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.523+0000 7f03fd400640 1 Processor -- start 2024-05-27T12:34:41.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.524+0000 7f03fd400640 1 -- start start 2024-05-27T12:34:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 0x7f03f81f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80d4f00 con 0x7f03f815eb00 2024-05-27T12:34:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80d4c20 con 0x7f03f80d3e50 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03fd400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80d51e0 con 0x7f03f80d1d70 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03f6800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.525+0000 7f03f5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49344/0 (socket says 172.21.15.97:49344) 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f5e00640 1 -- 172.21.15.97:0/3951399738 learned_addr learned my addr 172.21.15.97:0/3951399738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:59848/0 (socket says 172.21.15.97:59848) 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f7200640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 0x7f03f81f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 -- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 msgr2=0x7f03f81f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 -- 172.21.15.97:0/3951399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 msgr2=0x7f03f81f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 0x7f03f81f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 -- 172.21.15.97:0/3951399738 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f80cce80 con 0x7f03f80d1d70 2024-05-27T12:34:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f5e00640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f7200640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 0x7f03f81f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.526+0000 7f03f6800640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ba580 tx=0x7f03e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8041070 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e80306e0 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8037040 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f80cd160 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f80cd720 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03e8046450 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.527+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f03e803d840 con 0x7f03f80d1d70 2024-05-27T12:34:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.528+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8068a70 con 0x7f03f80d1d70 2024-05-27T12:34:41.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.533+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f03e803d420 con 0x7f03f80d1d70 2024-05-27T12:34:41.535 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:41.536+0000 7f021640e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T12:34:41.539 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:41.540+0000 7fe699121040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T12:34:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.610+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "90511b5d-5c7a-4221-b94f-5d9053d3f15c", "id": 2} v 0) -- 0x7f03f806bcc0 con 0x7f03f80d1d70 2024-05-27T12:34:41.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.615+0000 7f03eea00640 1 -- 172.21.15.97:0/3951399738 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90511b5d-5c7a-4221-b94f-5d9053d3f15c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f03e803dad0 con 0x7f03f80d1d70 2024-05-27T12:34:41.614 INFO:teuthology.orchestra.run.smithi097.stdout:2 2024-05-27T12:34:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 msgr2=0x7f03f80cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ba580 tx=0x7f03e80046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 shutdown_connections 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f80d3e50 0x7f03f81f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03f815eb00 0x7f03f81f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 --2- 172.21.15.97:0/3951399738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f80d1d70 0x7f03f80cdf20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 >> 172.21.15.97:0/3951399738 conn(0x7f03f8067a50 msgr2=0x7f03f80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 shutdown_connections 2024-05-27T12:34:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.616+0000 7f03fd400640 1 -- 172.21.15.97:0/3951399738 wait complete. 2024-05-27T12:34:41.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074 3 2024-05-27T12:34:41.642 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:41.643+0000 7f021640e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T12:34:41.651 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:41.652+0000 7fe699121040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T12:34:41.692 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:41.693+0000 7f021640e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T12:34:41.705 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:41.706+0000 7fe699121040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T12:34:41.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.811+0000 7fe7a7200640 1 Processor -- start 2024-05-27T12:34:41.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.811+0000 7fe7a7200640 1 -- start start 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.812+0000 7fe7a7200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe7a00d14a0 0x7fe7a00d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.812+0000 7fe7a7200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a0162ef0 con 0x7fe7a015e990 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.812+0000 7fe7a7200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a01631d0 con 0x7fe7a00d31b0 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.812+0000 7fe7a7200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a0162cb0 con 0x7fe7a00d14a0 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.813+0000 7fe7a6800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7a00d31b0 0x7fe7a00d3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38014/0 (socket says 172.21.15.97:38014) 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.813+0000 7fe7a6800640 1 -- 172.21.15.97:0/1409247185 learned_addr learned my addr 172.21.15.97:0/1409247185 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-05-27T12:34:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.813+0000 7fe7a5400640 1 --2- 172.21.15.97:0/1409247185 >> v2:172.21.15.94:3300/0 conn(0x7fe7a00d14a0 0x7fe7a00d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.813+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621384521 0 0) 0x7fe7a01631d0 con 0x7fe7a00d31b0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.813+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe788003520 con 0x7fe7a00d31b0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2368680071 0 0) 0x7fe788003520 con 0x7fe7a00d31b0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 >> v2:172.21.15.94:3300/0 conn(0x7fe7a00d14a0 msgr2=0x7fe7a00d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 --2- 172.21.15.97:0/1409247185 >> v2:172.21.15.94:3300/0 conn(0x7fe7a00d14a0 0x7fe7a00d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 >> v1:172.21.15.94:6789/0 conn(0x7fe7a015e990 legacy=0x7fe7a01625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00d29b0 con 0x7fe7a00d31b0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fe79c0019f0 con 0x7fe7a00d31b0 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 >> v1:172.21.15.97:6789/0 conn(0x7fe7a00d31b0 legacy=0x7fe7a00d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe788003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe788004800 0x7fe788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe788005210 0x7fe78800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0162cb0 con 0x7fe788004800 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe78800c160 con 0x7fe788003a00 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe78800c360 con 0x7fe788005210 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe788003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a6800640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe788005210 0x7fe78800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5400640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe788004800 0x7fe788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 -- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe788005210 msgr2=0x7fe78800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe788005210 0x7fe78800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 -- 172.21.15.97:0/1409247185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe788004800 msgr2=0x7fe788004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe788004800 0x7fe788004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.814+0000 7fe7a5e00640 1 -- 172.21.15.97:0/1409247185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe788003520 con 0x7fe788003a00 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.815+0000 7fe7a6800640 1 -- 172.21.15.97:0/1409247185 reap_dead start 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.815+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe788003e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe794001720 tx=0x7fe788005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50bbda5a770ac712 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794001370 con 0x7fe788003a00 2024-05-27T12:34:41.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7940052d0 con 0x7fe788003a00 2024-05-27T12:34:41.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a4a00640 1 -- 172.21.15.97:0/1409247185 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79402abc0 con 0x7fe788003a00 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a7200640 1 -- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 msgr2=0x7fe788003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a7200640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe788003e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe794001720 tx=0x7fe788005830 comp rx=0 tx=0).stop 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a7200640 1 -- 172.21.15.97:0/1409247185 shutdown_connections 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a7200640 1 --2- 172.21.15.97:0/1409247185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe788003e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.816+0000 7fe7a7200640 1 -- 172.21.15.97:0/1409247185 >> 172.21.15.97:0/1409247185 conn(0x7fe7a006cf90 msgr2=0x7fe7a006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.817+0000 7fe7a7200640 1 -- 172.21.15.97:0/1409247185 shutdown_connections 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.817+0000 7fe7a7200640 1 -- 172.21.15.97:0/1409247185 wait complete. 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.818+0000 7fe7a7200640 1 Processor -- start 2024-05-27T12:34:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.818+0000 7fe7a7200640 1 -- start start 2024-05-27T12:34:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7a0160370 0x7fe7a01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a0153b80 0x7fe7a0154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a015ed70 con 0x7fe7a0153b80 2024-05-27T12:34:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a01634b0 con 0x7fe788003a00 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.819+0000 7fe7a7200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a015eef0 con 0x7fe7a0160370 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37470/0 (socket says 172.21.15.97:37470) 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 -- 172.21.15.97:0/2696495980 learned_addr learned my addr 172.21.15.97:0/2696495980 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5400640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7a0160370 0x7fe7a01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 -- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7a0160370 msgr2=0x7fe7a01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7a0160370 0x7fe7a01526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a6800640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a0153b80 0x7fe7a0154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 -- 172.21.15.97:0/2696495980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a0153b80 msgr2=0x7fe7a0154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a0153b80 0x7fe7a0154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 -- 172.21.15.97:0/2696495980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00cce20 con 0x7fe788003a00 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.820+0000 7fe7a5e00640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe794001840 tx=0x7fe794001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79403f020 con 0x7fe788003a00 2024-05-27T12:34:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794005480 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794037040 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a00cd7c0 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a00cdd80 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe794042020 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.821+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe794003da0 con 0x7fe788003a00 2024-05-27T12:34:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.822+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a0075460 con 0x7fe788003a00 2024-05-27T12:34:41.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.824+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe794049030 con 0x7fe788003a00 2024-05-27T12:34:41.885 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:41.885+0000 7f021640e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T12:34:41.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.903+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074", "id": 3} v 0) -- 0x7fe7a00d23a0 con 0x7fe788003a00 2024-05-27T12:34:41.904 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:41.905+0000 7fe699121040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T12:34:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.908+0000 7fe79a000640 1 -- 172.21.15.97:0/2696495980 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe7a00cce20 con 0x7fe788003a00 2024-05-27T12:34:41.907 INFO:teuthology.orchestra.run.smithi097.stdout:3 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 msgr2=0x7fe7a00cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe794001840 tx=0x7fe794001120 comp rx=0 tx=0).stop 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 shutdown_connections 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a0153b80 0x7fe7a0154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7a0160370 0x7fe7a01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 --2- 172.21.15.97:0/2696495980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe788003a00 0x7fe7a00cd150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 >> 172.21.15.97:0/2696495980 conn(0x7fe7a006cf90 msgr2=0x7fe7a00b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 shutdown_connections 2024-05-27T12:34:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:41.910+0000 7fe7a7200640 1 -- 172.21.15.97:0/2696495980 wait complete. 2024-05-27T12:34:41.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5 4 2024-05-27T12:34:41.954 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:41.954+0000 7f021640e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T12:34:41.972 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:41.970+0000 7fe699121040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T12:34:42.014 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:42.015+0000 7f021640e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T12:34:42.034 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:42.035+0000 7fe699121040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T12:34:42.142 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:42.142+0000 7f021640e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T12:34:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.154+0000 7f8ea3e00640 1 Processor -- start 2024-05-27T12:34:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.154+0000 7f8ea3e00640 1 -- start start 2024-05-27T12:34:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 0x7f8ea40d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea3e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea4162ef0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea3e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea41631d0 con 0x7f8ea415e990 2024-05-27T12:34:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea3e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea4162cb0 con 0x7f8ea40d18e0 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea2a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ea415e990 0x7f8ea41625a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38026/0 (socket says 172.21.15.97:38026) 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea2a00640 1 -- 172.21.15.97:0/1173980721 learned_addr learned my addr 172.21.15.97:0/1173980721 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.156+0000 7f8ea3400640 1 --2- 172.21.15.97:0/1173980721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 0x7f8ea40d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.157+0000 7f8ea3400640 1 -- 172.21.15.97:0/1173980721 >> v1:172.21.15.97:6789/0 conn(0x7f8ea415e990 legacy=0x7f8ea41625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.157+0000 7f8ea3400640 1 -- 172.21.15.97:0/1173980721 >> v1:172.21.15.94:6789/0 conn(0x7f8ea40d14a0 legacy=0x7f8ea40d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.157+0000 7f8ea3400640 1 -- 172.21.15.97:0/1173980721 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea40d29b0 con 0x7f8ea40d18e0 2024-05-27T12:34:42.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.157+0000 7f8ea2000640 1 -- 172.21.15.97:0/1173980721 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788758603 0 0) 0x7f8ea41631d0 con 0x7f8ea415e990 2024-05-27T12:34:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.158+0000 7f8ea3400640 1 --2- 172.21.15.97:0/1173980721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 0x7f8ea40d38c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009dc0 tx=0x7f8e9c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83c7bb234cbe9dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.158+0000 7f8ea2000640 1 -- 172.21.15.97:0/1173980721 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c007740 con 0x7f8ea40d18e0 2024-05-27T12:34:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.158+0000 7f8ea2000640 1 -- 172.21.15.97:0/1173980721 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0078e0 con 0x7f8ea40d18e0 2024-05-27T12:34:42.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.158+0000 7f8ea2000640 1 -- 172.21.15.97:0/1173980721 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c007cb0 con 0x7f8ea40d18e0 2024-05-27T12:34:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 -- 172.21.15.97:0/1173980721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 msgr2=0x7f8ea40d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/1173980721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 0x7f8ea40d38c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009dc0 tx=0x7f8e9c02a5f0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 -- 172.21.15.97:0/1173980721 shutdown_connections 2024-05-27T12:34:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/1173980721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d18e0 0x7f8ea40d38c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 -- 172.21.15.97:0/1173980721 >> 172.21.15.97:0/1173980721 conn(0x7f8ea406cf90 msgr2=0x7f8ea406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 -- 172.21.15.97:0/1173980721 shutdown_connections 2024-05-27T12:34:42.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.159+0000 7f8ea3e00640 1 -- 172.21.15.97:0/1173980721 wait complete. 2024-05-27T12:34:42.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.161+0000 7f8ea3e00640 1 Processor -- start 2024-05-27T12:34:42.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.161+0000 7f8ea3e00640 1 -- start start 2024-05-27T12:34:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 0x7f8ea413c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea415e990 0x7f8ea413c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40d1d70 con 0x7f8ea40d18e0 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41634b0 con 0x7f8ea415e990 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea415ed70 con 0x7f8ea40d14a0 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea415e990 0x7f8ea413c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:47918/0 (socket says 172.21.15.97:47918) 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 -- 172.21.15.97:0/876883036 learned_addr learned my addr 172.21.15.97:0/876883036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8e9be00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 0x7f8ea413c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 -- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea415e990 msgr2=0x7f8ea413c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea415e990 0x7f8ea413c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 -- 172.21.15.97:0/876883036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 msgr2=0x7f8ea413c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 0x7f8ea413c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8ea2a00640 1 -- 172.21.15.97:0/876883036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea415cbe0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.162+0000 7f8e9be00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 0x7f8ea413c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:42.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8ea2a00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c500 tx=0x7f8e9400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94018070 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e94007580 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94004a20 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea415cec0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea415d480 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e94004bc0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.163+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8e94012910 con 0x7f8ea40d14a0 2024-05-27T12:34:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.164+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea4075460 con 0x7f8ea40d14a0 2024-05-27T12:34:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.167+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8e94012400 con 0x7f8ea40d14a0 2024-05-27T12:34:42.168 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:42.170+0000 7fe699121040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T12:34:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.246+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5", "id": 4} v 0) -- 0x7f8ea40d23a0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.251+0000 7f8e9aa00640 1 -- 172.21.15.97:0/876883036 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8e940125e0 con 0x7f8ea40d14a0 2024-05-27T12:34:42.251 INFO:teuthology.orchestra.run.smithi097.stdout:4 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 msgr2=0x7f8ea41f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c500 tx=0x7f8e9400c9c0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 shutdown_connections 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea415e990 0x7f8ea413c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ea40d18e0 0x7f8ea413c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 --2- 172.21.15.97:0/876883036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea40d14a0 0x7f8ea41f4460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 >> 172.21.15.97:0/876883036 conn(0x7f8ea406cf90 msgr2=0x7f8ea40b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 shutdown_connections 2024-05-27T12:34:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.253+0000 7f8ea3e00640 1 -- 172.21.15.97:0/876883036 wait complete. 2024-05-27T12:34:42.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new d9c76658-c9fa-4ef3-acf1-45b2e173b8f9 5 2024-05-27T12:34:42.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.436+0000 7f5d9a200640 1 Processor -- start 2024-05-27T12:34:42.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.436+0000 7f5d9a200640 1 -- start start 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d9a200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d9a200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 0x7f5d940d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d9a200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d940b7840 con 0x7f5d9415e900 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d9a200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5d94067f30 con 0x7f5d940b66d0 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d9a200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5d94068210 con 0x7f5d940ba3f0 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:47930/0 (socket says 172.21.15.97:47930) 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d93e00640 1 -- 172.21.15.97:0/1025748253 learned_addr learned my addr 172.21.15.97:0/1025748253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.438+0000 7f5d99800640 1 --2- 172.21.15.97:0/1025748253 >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 0x7f5d940d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.439+0000 7f5d93e00640 1 -- 172.21.15.97:0/1025748253 >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 msgr2=0x7f5d940d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.439+0000 7f5d93e00640 1 --2- 172.21.15.97:0/1025748253 >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 0x7f5d940d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.439+0000 7f5d93e00640 1 -- 172.21.15.97:0/1025748253 >> v1:172.21.15.94:6789/0 conn(0x7f5d9415e900 legacy=0x7f5d94067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.439+0000 7f5d93e00640 1 -- 172.21.15.97:0/1025748253 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d940684f0 con 0x7f5d940ba3f0 2024-05-27T12:34:42.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.439+0000 7f5d99800640 1 --2- 172.21.15.97:0/1025748253 >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 0x7f5d940d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.440+0000 7f5d93e00640 1 --2- 172.21.15.97:0/1025748253 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009870 tx=0x7f5d8002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c39bbf173f74e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.440+0000 7f5d93400640 1 -- 172.21.15.97:0/1025748253 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80037070 con 0x7f5d940ba3f0 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.440+0000 7f5d93400640 1 -- 172.21.15.97:0/1025748253 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80032440 con 0x7f5d940ba3f0 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.440+0000 7f5d93400640 1 -- 172.21.15.97:0/1025748253 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80032810 con 0x7f5d940ba3f0 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 -- 172.21.15.97:0/1025748253 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 msgr2=0x7f5d940b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 --2- 172.21.15.97:0/1025748253 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009870 tx=0x7f5d8002a290 comp rx=0 tx=0).stop 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 -- 172.21.15.97:0/1025748253 shutdown_connections 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 --2- 172.21.15.97:0/1025748253 >> v2:172.21.15.94:3300/0 conn(0x7f5d940b66d0 0x7f5d940d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 --2- 172.21.15.97:0/1025748253 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d940ba3f0 0x7f5d940b6190 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 -- 172.21.15.97:0/1025748253 >> 172.21.15.97:0/1025748253 conn(0x7f5d940d1190 msgr2=0x7f5d940d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 -- 172.21.15.97:0/1025748253 shutdown_connections 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.441+0000 7f5d9a200640 1 -- 172.21.15.97:0/1025748253 wait complete. 2024-05-27T12:34:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.443+0000 7f5d9a200640 1 Processor -- start 2024-05-27T12:34:42.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.443+0000 7f5d9a200640 1 -- start start 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d940b66d0 0x7f5d94157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 0x7f5d9415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9415ece0 con 0x7f5d940b66d0 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940d2df0 con 0x7f5d940ba3f0 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d9a200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5d940d3850 con 0x7f5d9415e900 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d99800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 0x7f5d9415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37482/0 (socket says 172.21.15.97:37482) 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 -- 172.21.15.97:0/3576474091 learned_addr learned my addr 172.21.15.97:0/3576474091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d98e00640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d940b66d0 0x7f5d94157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 -- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 msgr2=0x7f5d9415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 0x7f5d9415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 -- 172.21.15.97:0/3576474091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d940b66d0 msgr2=0x7f5d94157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d940b66d0 0x7f5d94157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d93e00640 1 -- 172.21.15.97:0/3576474091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9414b060 con 0x7f5d940ba3f0 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.444+0000 7f5d99800640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 0x7f5d9415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d93e00640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009870 tx=0x7f5d800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80041020 con 0x7f5d940ba3f0 2024-05-27T12:34:42.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80030970 con 0x7f5d940ba3f0 2024-05-27T12:34:42.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80037040 con 0x7f5d940ba3f0 2024-05-27T12:34:42.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d940cd6b0 con 0x7f5d940ba3f0 2024-05-27T12:34:42.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d940cdc70 con 0x7f5d940ba3f0 2024-05-27T12:34:42.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.445+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5d80044020 con 0x7f5d940ba3f0 2024-05-27T12:34:42.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.446+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5d80005d20 con 0x7f5d940ba3f0 2024-05-27T12:34:42.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.446+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d94075460 con 0x7f5d940ba3f0 2024-05-27T12:34:42.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.449+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5d8004b070 con 0x7f5d940ba3f0 2024-05-27T12:34:42.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.527+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9c76658-c9fa-4ef3-acf1-45b2e173b8f9", "id": 5} v 0) -- 0x7f5d940d1710 con 0x7f5d940ba3f0 2024-05-27T12:34:42.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.534+0000 7f5d90c00640 1 -- 172.21.15.97:0/3576474091 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9c76658-c9fa-4ef3-acf1-45b2e173b8f9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5d940b7840 con 0x7f5d940ba3f0 2024-05-27T12:34:42.533 INFO:teuthology.orchestra.run.smithi097.stdout:5 2024-05-27T12:34:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.535+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 msgr2=0x7f5d9415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.535+0000 7f5d9a200640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009870 tx=0x7f5d800046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.535+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 shutdown_connections 2024-05-27T12:34:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.535+0000 7f5d9a200640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940ba3f0 0x7f5d9415f800 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.535+0000 7f5d9a200640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d9415e900 0x7f5d9415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.536+0000 7f5d9a200640 1 --2- 172.21.15.97:0/3576474091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d940b66d0 0x7f5d94157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.536+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 >> 172.21.15.97:0/3576474091 conn(0x7f5d940d1190 msgr2=0x7f5d940cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.536+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 shutdown_connections 2024-05-27T12:34:42.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.536+0000 7f5d9a200640 1 -- 172.21.15.97:0/3576474091 wait complete. 2024-05-27T12:34:42.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6 6 2024-05-27T12:34:42.602 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:42.603+0000 7f021640e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T12:34:42.650 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:42.651+0000 7fe699121040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T12:34:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.729+0000 7fdb07e00640 1 Processor -- start 2024-05-27T12:34:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.729+0000 7fdb07e00640 1 -- start start 2024-05-27T12:34:42.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb081625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 0x7fdb080d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb08162ef0 con 0x7fdb080d14a0 2024-05-27T12:34:42.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdb081634b0 con 0x7fdb080d41c0 2024-05-27T12:34:42.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdb080d1e20 con 0x7fdb0815e990 2024-05-27T12:34:42.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb07400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 0x7fdb080d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdaffe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdb080d14a0 0x7fdb080b61b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38042/0 (socket says 172.21.15.97:38042) 2024-05-27T12:34:42.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdaffe00640 1 -- 172.21.15.97:0/799823385 learned_addr learned my addr 172.21.15.97:0/799823385 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-05-27T12:34:42.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.731+0000 7fdb06a00640 1 --2- 172.21.15.97:0/799823385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb081625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06000640 1 -- 172.21.15.97:0/799823385 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829502981 0 0) 0x7fdb08162ef0 con 0x7fdb080d14a0 2024-05-27T12:34:42.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06000640 1 -- 172.21.15.97:0/799823385 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaec003520 con 0x7fdb080d14a0 2024-05-27T12:34:42.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06a00640 1 -- 172.21.15.97:0/799823385 >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 msgr2=0x7fdb080d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:42.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06a00640 1 --2- 172.21.15.97:0/799823385 >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 0x7fdb080d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06a00640 1 -- 172.21.15.97:0/799823385 >> v1:172.21.15.97:6789/0 conn(0x7fdb080d14a0 legacy=0x7fdb080b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb06a00640 1 -- 172.21.15.97:0/799823385 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb081631d0 con 0x7fdb0815e990 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.732+0000 7fdb07400640 1 --2- 172.21.15.97:0/799823385 >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 0x7fdb080d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.733+0000 7fdb06a00640 1 --2- 172.21.15.97:0/799823385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb081625a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097a0 tx=0x7fdaf802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d45a7dedb3ffe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.733+0000 7fdb06000640 1 -- 172.21.15.97:0/799823385 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8037070 con 0x7fdb0815e990 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.733+0000 7fdb06000640 1 -- 172.21.15.97:0/799823385 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf802ae40 con 0x7fdb0815e990 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.733+0000 7fdb06000640 1 -- 172.21.15.97:0/799823385 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8032640 con 0x7fdb0815e990 2024-05-27T12:34:42.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 -- 172.21.15.97:0/799823385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 msgr2=0x7fdb081625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 --2- 172.21.15.97:0/799823385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb081625a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80097a0 tx=0x7fdaf802a0f0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 -- 172.21.15.97:0/799823385 shutdown_connections 2024-05-27T12:34:42.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 --2- 172.21.15.97:0/799823385 >> v2:172.21.15.97:3300/0 conn(0x7fdb080d41c0 0x7fdb080d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 --2- 172.21.15.97:0/799823385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb081625a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 -- 172.21.15.97:0/799823385 >> 172.21.15.97:0/799823385 conn(0x7fdb0806cf90 msgr2=0x7fdb0806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 -- 172.21.15.97:0/799823385 shutdown_connections 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.734+0000 7fdb07e00640 1 -- 172.21.15.97:0/799823385 wait complete. 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.736+0000 7fdb07e00640 1 Processor -- start 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.736+0000 7fdb07e00640 1 -- start start 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb080d41c0 0x7fdb0813c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb0813c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:42.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0815ed70 con 0x7fdb080d41c0 2024-05-27T12:34:42.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb080d2c90 con 0x7fdb080d14a0 2024-05-27T12:34:42.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0815eef0 con 0x7fdb0815e990 2024-05-27T12:34:42.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37500/0 (socket says 172.21.15.97:37500) 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 -- 172.21.15.97:0/4173790380 learned_addr learned my addr 172.21.15.97:0/4173790380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb07400640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb0813c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdaffe00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb080d41c0 0x7fdb0813c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 -- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 msgr2=0x7fdb0813c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb0813c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 -- 172.21.15.97:0/4173790380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb080d41c0 msgr2=0x7fdb0813c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb080d41c0 0x7fdb0813c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.737+0000 7fdb06a00640 1 -- 172.21.15.97:0/4173790380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb080cbdf0 con 0x7fdb080d14a0 2024-05-27T12:34:42.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdb06a00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802a6e0 tx=0x7fdaf80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:42.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8040030 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf8038040 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8037070 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb08151160 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb08151720 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.738+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdaf8043070 con 0x7fdb080d14a0 2024-05-27T12:34:42.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.739+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fdaf8045410 con 0x7fdb080d14a0 2024-05-27T12:34:42.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.739+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb080d0040 con 0x7fdb080d14a0 2024-05-27T12:34:42.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.742+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdb080d0040 con 0x7fdb080d14a0 2024-05-27T12:34:42.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.822+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6", "id": 6} v 0) -- 0x7fdb08075460 con 0x7fdb080d14a0 2024-05-27T12:34:42.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.830+0000 7fdafea00640 1 -- 172.21.15.97:0/4173790380 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fdaf8032440 con 0x7fdb080d14a0 2024-05-27T12:34:42.830 INFO:teuthology.orchestra.run.smithi097.stdout:6 2024-05-27T12:34:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 msgr2=0x7fdb080cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802a6e0 tx=0x7fdaf80046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 shutdown_connections 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdb0815e990 0x7fdb0813c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb080d41c0 0x7fdb0813c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 --2- 172.21.15.97:0/4173790380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb080d14a0 0x7fdb080cb8b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 >> 172.21.15.97:0/4173790380 conn(0x7fdb0806cf90 msgr2=0x7fdb080c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 shutdown_connections 2024-05-27T12:34:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:42.832+0000 7fdb07e00640 1 -- 172.21.15.97:0/4173790380 wait complete. 2024-05-27T12:34:42.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new bc92dcb4-e3cd-4c79-818c-68ee1160d690 7 2024-05-27T12:34:42.904 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:42.904+0000 7f021640e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T12:34:42.963 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:42.964+0000 7fe699121040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T12:34:42.965 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:42.965+0000 7f021640e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T12:34:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.015+0000 7f5ebbe00640 1 Processor -- start 2024-05-27T12:34:43.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.015+0000 7f5ebbe00640 1 -- start start 2024-05-27T12:34:43.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.016+0000 7f5ebbe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f5ebc15e900 0x7f5ebc067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.016+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.016+0000 7f5ebbe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0b7840 con 0x7f5ebc0ba3f0 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.016+0000 7f5ebbe00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc067f30 con 0x7f5ebc15e900 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.016+0000 7f5ebbe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc068210 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:47976/0 (socket says 172.21.15.97:47976) 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 -- 172.21.15.97:0/408933315 learned_addr learned my addr 172.21.15.97:0/408933315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebaa00640 1 --2- 172.21.15.97:0/408933315 >> v2:172.21.15.94:3300/0 conn(0x7f5ebc15e900 0x7f5ebc067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 -- 172.21.15.97:0/408933315 >> v2:172.21.15.94:3300/0 conn(0x7f5ebc15e900 msgr2=0x7f5ebc067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 --2- 172.21.15.97:0/408933315 >> v2:172.21.15.94:3300/0 conn(0x7f5ebc15e900 0x7f5ebc067630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 -- 172.21.15.97:0/408933315 >> v1:172.21.15.97:6789/0 conn(0x7f5ebc0ba3f0 legacy=0x7f5ebc0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5ebb400640 1 -- 172.21.15.97:0/408933315 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0684f0 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.017+0000 7f5eba000640 1 -- 172.21.15.97:0/408933315 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845517529 0 0) 0x7f5ea4001110 con 0x7f5ebc0ba3f0 2024-05-27T12:34:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.018+0000 7f5ebb400640 1 --2- 172.21.15.97:0/408933315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6bca99f3d7cc8f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.018+0000 7f5eba000640 1 -- 172.21.15.97:0/408933315 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4037070 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.018+0000 7f5eba000640 1 -- 172.21.15.97:0/408933315 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb402ae50 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.018+0000 7f5eba000640 1 -- 172.21.15.97:0/408933315 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4032860 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 -- 172.21.15.97:0/408933315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 msgr2=0x7f5ebc0d3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/408933315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb402a300 comp rx=0 tx=0).stop 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 -- 172.21.15.97:0/408933315 shutdown_connections 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/408933315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc0b68a0 0x7f5ebc0d3310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/408933315 >> v2:172.21.15.94:3300/0 conn(0x7f5ebc15e900 0x7f5ebc067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 -- 172.21.15.97:0/408933315 >> 172.21.15.97:0/408933315 conn(0x7f5ebc0d1190 msgr2=0x7f5ebc0d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 -- 172.21.15.97:0/408933315 shutdown_connections 2024-05-27T12:34:43.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.019+0000 7f5ebbe00640 1 -- 172.21.15.97:0/408933315 wait complete. 2024-05-27T12:34:43.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.021+0000 7f5ebbe00640 1 Processor -- start 2024-05-27T12:34:43.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.021+0000 7f5ebbe00640 1 -- start start 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ebc0b68a0 0x7f5ebc15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ebc0ba3f0 0x7f5ebc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc15ece0 con 0x7f5ebc0ba3f0 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d2df0 con 0x7f5ebc0b68a0 2024-05-27T12:34:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebbe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc067b70 con 0x7f5ebc15e900 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ebc0b68a0 0x7f5ebc15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:47982/0 (socket says 172.21.15.97:47982) 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.022+0000 7f5ebb400640 1 -- 172.21.15.97:0/4230633097 learned_addr learned my addr 172.21.15.97:0/4230633097 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5eb2000640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ebc0ba3f0 0x7f5ebc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 -- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ebc0b68a0 msgr2=0x7f5ebc15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ebc0b68a0 0x7f5ebc15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 -- 172.21.15.97:0/4230633097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ebc0ba3f0 msgr2=0x7f5ebc15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ebc0ba3f0 0x7f5ebc15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 -- 172.21.15.97:0/4230633097 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1f4780 con 0x7f5ebc15e900 2024-05-27T12:34:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebb400640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4041070 con 0x7f5ebc15e900 2024-05-27T12:34:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb403c680 con 0x7f5ebc15e900 2024-05-27T12:34:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc1f4a60 con 0x7f5ebc15e900 2024-05-27T12:34:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4037040 con 0x7f5ebc15e900 2024-05-27T12:34:43.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.023+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1f5020 con 0x7f5ebc15e900 2024-05-27T12:34:43.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.024+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5eb4044070 con 0x7f5ebc15e900 2024-05-27T12:34:43.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.024+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f5eb403da50 con 0x7f5ebc15e900 2024-05-27T12:34:43.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.024+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0cfff0 con 0x7f5ebc15e900 2024-05-27T12:34:43.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.027+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5ebc0cfff0 con 0x7f5ebc15e900 2024-05-27T12:34:43.028 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.029+0000 7fe699121040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T12:34:43.097 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.097+0000 7f021640e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T12:34:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.105+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bc92dcb4-e3cd-4c79-818c-68ee1160d690", "id": 7} v 0) -- 0x7f5ebc075460 con 0x7f5ebc15e900 2024-05-27T12:34:43.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.115+0000 7f5eb3400640 1 -- 172.21.15.97:0/4230633097 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc92dcb4-e3cd-4c79-818c-68ee1160d690", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f5eb4046450 con 0x7f5ebc15e900 2024-05-27T12:34:43.115 INFO:teuthology.orchestra.run.smithi097.stdout:7 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 msgr2=0x7f5ebc15fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb40046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 shutdown_connections 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ebc15e900 0x7f5ebc15fd60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ebc0ba3f0 0x7f5ebc15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 --2- 172.21.15.97:0/4230633097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ebc0b68a0 0x7f5ebc15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 >> 172.21.15.97:0/4230633097 conn(0x7f5ebc0d1190 msgr2=0x7f5ebc15dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 shutdown_connections 2024-05-27T12:34:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:34:43.116+0000 7f5ebbe00640 1 -- 172.21.15.97:0/4230633097 wait complete. 2024-05-27T12:34:43.132 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-27T12:34:43.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-27T12:34:43.140 INFO:tasks.ceph.osd.0:Started 2024-05-27T12:34:43.141 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-27T12:34:43.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-27T12:34:43.149 INFO:tasks.ceph.osd.1:Started 2024-05-27T12:34:43.149 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-27T12:34:43.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-27T12:34:43.155 INFO:tasks.ceph.osd.2:Started 2024-05-27T12:34:43.155 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-27T12:34:43.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-27T12:34:43.161 INFO:tasks.ceph.osd.3:Started 2024-05-27T12:34:43.161 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-27T12:34:43.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-27T12:34:43.168 INFO:tasks.ceph.osd.4:Started 2024-05-27T12:34:43.169 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-27T12:34:43.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-27T12:34:43.171 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.167+0000 7fe699121040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T12:34:43.177 INFO:tasks.ceph.osd.5:Started 2024-05-27T12:34:43.177 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-27T12:34:43.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-27T12:34:43.183 INFO:tasks.ceph.osd.6:Started 2024-05-27T12:34:43.183 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-27T12:34:43.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-27T12:34:43.189 INFO:tasks.ceph.osd.7:Started 2024-05-27T12:34:43.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:34:43.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.372+0000 7fcb20400640 1 Processor -- start 2024-05-27T12:34:43.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.373+0000 7fcb20400640 1 -- start start 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb20400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcb181620b0 0x7fcb180b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb20400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb180cf8d0 0x7fcb1806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb20400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb180b39a0 con 0x7fcb180cfd30 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb20400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcb180d56d0 con 0x7fcb181620b0 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb20400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180b7f00 con 0x7fcb180cf8d0 2024-05-27T12:34:43.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcb181620b0 0x7fcb180b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcb181620b0 0x7fcb180b79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53868/0 (socket says 172.21.15.94:53868) 2024-05-27T12:34:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 -- 172.21.15.94:0/1766360066 learned_addr learned my addr 172.21.15.94:0/1766360066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1d000640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb180cf8d0 0x7fcb1806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 -- 172.21.15.94:0/1766360066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb180cf8d0 msgr2=0x7fcb1806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb180cf8d0 0x7fcb1806ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 -- 172.21.15.94:0/1766360066 >> v1:172.21.15.94:6789/0 conn(0x7fcb180cfd30 legacy=0x7fcb180d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.374+0000 7fcb1da00640 1 -- 172.21.15.94:0/1766360066 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb180b3c80 con 0x7fcb181620b0 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb1da00640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb181620b0 0x7fcb180b79c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b58416ff13b062 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb17e00640 1 -- 172.21.15.94:0/1766360066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c037070 con 0x7fcb181620b0 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb17e00640 1 -- 172.21.15.94:0/1766360066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c032440 con 0x7fcb181620b0 2024-05-27T12:34:43.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb17e00640 1 -- 172.21.15.94:0/1766360066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c032810 con 0x7fcb181620b0 2024-05-27T12:34:43.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb20400640 1 -- 172.21.15.94:0/1766360066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb181620b0 msgr2=0x7fcb180b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.376+0000 7fcb20400640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb181620b0 0x7fcb180b79c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c02a250 comp rx=0 tx=0).stop 2024-05-27T12:34:43.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 -- 172.21.15.94:0/1766360066 shutdown_connections 2024-05-27T12:34:43.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb180cf8d0 0x7fcb1806ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 --2- 172.21.15.94:0/1766360066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb181620b0 0x7fcb180b79c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 -- 172.21.15.94:0/1766360066 >> 172.21.15.94:0/1766360066 conn(0x7fcb180d1940 msgr2=0x7fcb180d4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 -- 172.21.15.94:0/1766360066 shutdown_connections 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.377+0000 7fcb20400640 1 -- 172.21.15.94:0/1766360066 wait complete. 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.378+0000 7fcb20400640 1 Processor -- start 2024-05-27T12:34:43.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.378+0000 7fcb20400640 1 -- start start 2024-05-27T12:34:43.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb180cf8d0 0x7fcb180ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 0x7fcb181f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180d3430 con 0x7fcb180cfd30 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180d3150 con 0x7fcb180cf8d0 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb20400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180b3650 con 0x7fcb181620b0 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.379+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53882/0 (socket says 172.21.15.94:53882) 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 -- 172.21.15.94:0/486011920 learned_addr learned my addr 172.21.15.94:0/486011920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1da00640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb180cf8d0 0x7fcb180ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1e400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 0x7fcb181f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 -- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 msgr2=0x7fcb181f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 0x7fcb181f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 -- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb180cf8d0 msgr2=0x7fcb180ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb180cf8d0 0x7fcb180ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 -- 172.21.15.94:0/486011920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181f8d60 con 0x7fcb180cfd30 2024-05-27T12:34:43.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1e400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 0x7fcb181f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:34:43.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb1d000640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e870 tx=0x7fcb0400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:43.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.380+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04017070 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.381+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0400c940 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.381+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400cd30 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.381+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181f9040 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.381+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb18154b10 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.382+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb0401a070 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.382+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcb040065a0 con 0x7fcb180cfd30 2024-05-27T12:34:43.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.382+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb180d0200 con 0x7fcb180cfd30 2024-05-27T12:34:43.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.387+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcb180d0200 con 0x7fcb180cfd30 2024-05-27T12:34:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.449+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb18075460 con 0x7fcb180cfd30 2024-05-27T12:34:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.450+0000 7fcb15600640 1 -- 172.21.15.94:0/486011920 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcb040067f0 con 0x7fcb180cfd30 2024-05-27T12:34:43.449 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:43.449 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:43.107860+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:34:43.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 msgr2=0x7fcb181f8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:43.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e870 tx=0x7fcb0400ed30 comp rx=0 tx=0).stop 2024-05-27T12:34:43.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 shutdown_connections 2024-05-27T12:34:43.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb181620b0 0x7fcb181f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb180cfd30 0x7fcb181f8600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 --2- 172.21.15.94:0/486011920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb180cf8d0 0x7fcb180ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 >> 172.21.15.94:0/486011920 conn(0x7fcb180d1940 msgr2=0x7fcb180b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 shutdown_connections 2024-05-27T12:34:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:43.451+0000 7fcb20400640 1 -- 172.21.15.94:0/486011920 wait complete. 2024-05-27T12:34:43.465 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-27T12:34:43.466 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-27T12:34:43.515 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.515+0000 7f021640e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T12:34:43.527 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.528+0000 7fe699121040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T12:34:43.571 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.572+0000 7f021640e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T12:34:43.583 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.584+0000 7fe699121040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T12:34:43.758 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.758+0000 7f021640e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T12:34:43.765 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.766+0000 7fe699121040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T12:34:43.830 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-05-27T12:34:43.830+0000 7f3305a63640 -1 Falling back to public interface 2024-05-27T12:34:43.864 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-05-27T12:34:43.865+0000 7f13a9add640 -1 Falling back to public interface 2024-05-27T12:34:43.866 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-05-27T12:34:43.867+0000 7f4e70863640 -1 Falling back to public interface 2024-05-27T12:34:43.868 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-05-27T12:34:43.869+0000 7fc82a018640 -1 Falling back to public interface 2024-05-27T12:34:43.878 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-05-27T12:34:43.879+0000 7f0cae041640 -1 Falling back to public interface 2024-05-27T12:34:43.879 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-05-27T12:34:43.881+0000 7ff5f19cb640 -1 Falling back to public interface 2024-05-27T12:34:43.881 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-05-27T12:34:43.882+0000 7fcee7010640 -1 Falling back to public interface 2024-05-27T12:34:43.882 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-05-27T12:34:43.883+0000 7f9723a0f640 -1 Falling back to public interface 2024-05-27T12:34:43.937 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.937+0000 7f021640e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T12:34:43.945 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:43.946+0000 7fe699121040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T12:34:43.995 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:43.996+0000 7f021640e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T12:34:44.002 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:44.003+0000 7fe699121040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T12:34:44.076 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:44.077+0000 7f021640e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T12:34:44.082 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:44.083+0000 7fe699121040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T12:34:44.130 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:44.131+0000 7f021640e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T12:34:44.135 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:44.136+0000 7fe699121040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T12:34:44.272 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T12:34:44.429 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.431+0000 7f048e600640 1 Processor -- start 2024-05-27T12:34:44.429 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.431+0000 7f048e600640 1 -- start start 2024-05-27T12:34:44.431 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f048e600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f048815ea30 0x7f04881625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:44.431 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f048e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 0x7f04880cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:44.432 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f048e600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04880ba880 con 0x7f04880cfb50 2024-05-27T12:34:44.432 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f048e600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0488163540 con 0x7f048815ea30 2024-05-27T12:34:44.432 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f048e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f04880d2f00 con 0x7f04880ba420 2024-05-27T12:34:44.432 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f0487e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f04880cfb50 0x7f04880b63c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:35336/0 (socket says 172.21.15.94:35336) 2024-05-27T12:34:44.432 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.433+0000 7f0487e00640 1 -- 172.21.15.94:0/1185441453 learned_addr learned my addr 172.21.15.94:0/1185441453 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0487400640 1 --2- 172.21.15.94:0/1185441453 >> v2:172.21.15.97:3300/0 conn(0x7f048815ea30 0x7f04881625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0486a00640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 0x7f04880cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0486000640 1 -- 172.21.15.94:0/1185441453 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830498115 0 0) 0x7f04880ba880 con 0x7f04880cfb50 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0486000640 1 -- 172.21.15.94:0/1185441453 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f046c003520 con 0x7f04880cfb50 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0487400640 1 -- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 msgr2=0x7f04880cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0487400640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 0x7f04880cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0487400640 1 -- 172.21.15.94:0/1185441453 >> v1:172.21.15.94:6789/0 conn(0x7f04880cfb50 legacy=0x7f04880b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0487400640 1 -- 172.21.15.94:0/1185441453 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488162ca0 con 0x7f048815ea30 2024-05-27T12:34:44.433 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.434+0000 7f0486a00640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 0x7f04880cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.435+0000 7f0487400640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f048815ea30 0x7f04881625a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04780097a0 tx=0x7f047802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2221717bfabde827 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.435+0000 7f0486000640 1 -- 172.21.15.94:0/1185441453 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478037070 con 0x7f048815ea30 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f0486000640 1 -- 172.21.15.94:0/1185441453 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047802ae50 con 0x7f048815ea30 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f0486000640 1 -- 172.21.15.94:0/1185441453 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478032660 con 0x7f048815ea30 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 -- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f048815ea30 msgr2=0x7f04881625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f048815ea30 0x7f04881625a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04780097a0 tx=0x7f047802a100 comp rx=0 tx=0).stop 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 -- 172.21.15.94:0/1185441453 shutdown_connections 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f04880ba420 0x7f04880cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.435 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 --2- 172.21.15.94:0/1185441453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f048815ea30 0x7f04881625a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.436 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 -- 172.21.15.94:0/1185441453 >> 172.21.15.94:0/1185441453 conn(0x7f048806cb50 msgr2=0x7f048806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:44.436 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 -- 172.21.15.94:0/1185441453 shutdown_connections 2024-05-27T12:34:44.436 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.436+0000 7f048e600640 1 -- 172.21.15.94:0/1185441453 wait complete. 2024-05-27T12:34:44.436 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.438+0000 7f048e600640 1 Processor -- start 2024-05-27T12:34:44.436 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.438+0000 7f048e600640 1 -- start start 2024-05-27T12:34:44.438 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:44.438 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04880cfb50 0x7f04880b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:44.438 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f048815ea30 0x7f04880d4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:44.438 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f048815ee10 con 0x7f04880ba420 2024-05-27T12:34:44.438 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0488163260 con 0x7f04880cfb50 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f048e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f04880d43a0 con 0x7f048815ea30 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f0487400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f0487400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53888/0 (socket says 172.21.15.94:53888) 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f0487400640 1 -- 172.21.15.94:0/4221748227 learned_addr learned my addr 172.21.15.94:0/4221748227 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.439+0000 7f0486a00640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04880cfb50 0x7f04880b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487e00640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f048815ea30 0x7f04880d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 -- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f048815ea30 msgr2=0x7f04880d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f048815ea30 0x7f04880d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 -- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04880cfb50 msgr2=0x7f04880b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04880cfb50 0x7f04880b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 -- 172.21.15.94:0/4221748227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488160110 con 0x7f04880ba420 2024-05-27T12:34:44.439 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f0487400640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0478035040 tx=0x7f04780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:44.440 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478041070 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047803c460 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478037040 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.440+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04881603f0 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.441+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04881609b0 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.441+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f047803a070 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.441+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0478045680 con 0x7f04880ba420 2024-05-27T12:34:44.441 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.442+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048804fce0 con 0x7f04880ba420 2024-05-27T12:34:44.442 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.444+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f047803c870 con 0x7f04880ba420 2024-05-27T12:34:44.510 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.512+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f048815e3a0 con 0x7f04880ba420 2024-05-27T12:34:44.511 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.512+0000 7f047f400640 1 -- 172.21.15.94:0/4221748227 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f047803ca50 con 0x7f04880ba420 2024-05-27T12:34:44.511 INFO:teuthology.misc.health.smithi094.stdout: 2024-05-27T12:34:44.511 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:43.107860+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:34:44.512 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 msgr2=0x7f0488155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:44.512 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0478035040 tx=0x7f04780046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 shutdown_connections 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04880cfb50 0x7f04880b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f048815ea30 0x7f04880d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 --2- 172.21.15.94:0/4221748227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04880ba420 0x7f0488155e80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 >> 172.21.15.94:0/4221748227 conn(0x7f048806cb50 msgr2=0x7f0488074e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 shutdown_connections 2024-05-27T12:34:44.513 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:44.514+0000 7f048e600640 1 -- 172.21.15.94:0/4221748227 wait complete. 2024-05-27T12:34:44.525 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-27T12:34:44.688 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:44.688+0000 7fe699121040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T12:34:44.691 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:44.693+0000 7f021640e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T12:34:44.823 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:44.824+0000 7fe699121040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T12:34:44.825 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-05-27T12:34:44.827+0000 7f021640e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T12:34:45.866 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:45.867+0000 7fe639800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-27T12:34:45.868 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:34:45.869+0000 7fe639800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-27T12:34:47.269 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-05-27T12:34:47.271+0000 7f0cae041640 -1 osd.3 0 log_to_monitors true 2024-05-27T12:34:47.413 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-05-27T12:34:47.415+0000 7f3305a63640 -1 osd.0 0 log_to_monitors true 2024-05-27T12:34:47.457 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-05-27T12:34:47.458+0000 7f4e70863640 -1 osd.1 0 log_to_monitors true 2024-05-27T12:34:47.483 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-05-27T12:34:47.484+0000 7fc82a018640 -1 osd.5 0 log_to_monitors true 2024-05-27T12:34:47.499 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-05-27T12:34:47.501+0000 7f9723a0f640 -1 osd.2 0 log_to_monitors true 2024-05-27T12:34:47.522 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-05-27T12:34:47.523+0000 7f13a9add640 -1 osd.7 0 log_to_monitors true 2024-05-27T12:34:47.564 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-05-27T12:34:47.565+0000 7ff5f19cb640 -1 osd.4 0 log_to_monitors true 2024-05-27T12:34:47.607 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-05-27T12:34:47.608+0000 7fcee7010640 -1 osd.6 0 log_to_monitors true 2024-05-27T12:34:50.649 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-05-27T12:34:50.650+0000 7f0ca8e00640 -1 osd.3 0 waiting for initial osdmap 2024-05-27T12:34:50.657 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-05-27T12:34:50.657+0000 7f0c9ee00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:50.678 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-05-27T12:34:50.679+0000 7f4e6b800640 -1 osd.1 0 waiting for initial osdmap 2024-05-27T12:34:50.685 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-05-27T12:34:50.685+0000 7f4e61800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:50.688 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-05-27T12:34:50.689+0000 7f971ec00640 -1 osd.2 0 waiting for initial osdmap 2024-05-27T12:34:50.695 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-05-27T12:34:50.696+0000 7f9714200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:50.736 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-05-27T12:34:50.737+0000 7f3300a00640 -1 osd.0 0 waiting for initial osdmap 2024-05-27T12:34:50.749 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-05-27T12:34:50.749+0000 7f32f6a00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:51.145 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-05-27T12:34:51.144+0000 7fcee2200640 -1 osd.6 0 waiting for initial osdmap 2024-05-27T12:34:51.150 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-05-27T12:34:51.151+0000 7fced7800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:51.174 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-05-27T12:34:51.174+0000 7f13a4a00640 -1 osd.7 0 waiting for initial osdmap 2024-05-27T12:34:51.182 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-05-27T12:34:51.183+0000 7f139aa00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:51.189 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-05-27T12:34:51.189+0000 7ff5ec800640 -1 osd.4 0 waiting for initial osdmap 2024-05-27T12:34:51.198 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-05-27T12:34:51.198+0000 7ff5e2800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:51.202 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-05-27T12:34:51.202+0000 7fc825400640 -1 osd.5 0 waiting for initial osdmap 2024-05-27T12:34:51.212 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-05-27T12:34:51.212+0000 7fc81aa00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T12:34:51.332 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T12:34:51.517 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.517+0000 7f035ee00640 1 Processor -- start 2024-05-27T12:34:51.517 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.517+0000 7f035ee00640 1 -- start start 2024-05-27T12:34:51.518 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ee00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03580b5650 0x7f03580d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.518 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ee00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f03580d2b20 0x7f03580cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.518 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ee00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03580d2f30 con 0x7f035815eb20 2024-05-27T12:34:51.519 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ee00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f035815ef00 con 0x7f03580d2b20 2024-05-27T12:34:51.519 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ee00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f035806dee0 con 0x7f03580b5650 2024-05-27T12:34:51.519 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ce00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f03580d2b20 0x7f03580cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ce00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f03580d2b20 0x7f03580cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54964/0 (socket says 172.21.15.94:54964) 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.519+0000 7f035ce00640 1 -- 172.21.15.94:0/2087466355 learned_addr learned my addr 172.21.15.94:0/2087466355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.520+0000 7f034f400640 1 --2- 172.21.15.94:0/2087466355 >> v2:172.21.15.97:3300/0 conn(0x7f03580b5650 0x7f03580d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.520+0000 7f035ce00640 1 -- 172.21.15.94:0/2087466355 >> v2:172.21.15.97:3300/0 conn(0x7f03580b5650 msgr2=0x7f03580d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.520+0000 7f035ce00640 1 --2- 172.21.15.94:0/2087466355 >> v2:172.21.15.97:3300/0 conn(0x7f03580b5650 0x7f03580d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.520+0000 7f035ce00640 1 -- 172.21.15.94:0/2087466355 >> v1:172.21.15.97:6789/0 conn(0x7f035815eb20 legacy=0x7f035806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.520 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.520+0000 7f035ce00640 1 -- 172.21.15.94:0/2087466355 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03580b5dd0 con 0x7f03580d2b20 2024-05-27T12:34:51.521 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.521+0000 7f035ce00640 1 --2- 172.21.15.94:0/2087466355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f03580cf8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f035806eb50 tx=0x7f035002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eab46dc42e350c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.521 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.522+0000 7f0357400640 1 -- 172.21.15.94:0/2087466355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350037070 con 0x7f03580d2b20 2024-05-27T12:34:51.521 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.522+0000 7f0357400640 1 -- 172.21.15.94:0/2087466355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0350032440 con 0x7f03580d2b20 2024-05-27T12:34:51.521 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.522+0000 7f0357400640 1 -- 172.21.15.94:0/2087466355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350032810 con 0x7f03580d2b20 2024-05-27T12:34:51.522 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.522+0000 7f035ee00640 1 -- 172.21.15.94:0/2087466355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 msgr2=0x7f03580cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.522 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.522+0000 7f035ee00640 1 --2- 172.21.15.94:0/2087466355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f03580cf8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f035806eb50 tx=0x7f035002a1e0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.522 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 -- 172.21.15.94:0/2087466355 shutdown_connections 2024-05-27T12:34:51.522 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 --2- 172.21.15.94:0/2087466355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f03580cf8c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.523 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 --2- 172.21.15.94:0/2087466355 >> v2:172.21.15.97:3300/0 conn(0x7f03580b5650 0x7f03580d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.523 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 -- 172.21.15.94:0/2087466355 >> 172.21.15.94:0/2087466355 conn(0x7f03580b4350 msgr2=0x7f03580b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:51.523 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 -- 172.21.15.94:0/2087466355 shutdown_connections 2024-05-27T12:34:51.523 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.523+0000 7f035ee00640 1 -- 172.21.15.94:0/2087466355 wait complete. 2024-05-27T12:34:51.524 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.525+0000 7f035ee00640 1 Processor -- start 2024-05-27T12:34:51.524 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.525+0000 7f035ee00640 1 -- start start 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03580b5650 0x7f03581422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f035815eb20 0x7f035815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03580b6570 con 0x7f03580d2b20 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03580b6290 con 0x7f03580b5650 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f035ee00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f03580b6850 con 0x7f035815eb20 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.526 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54976/0 (socket says 172.21.15.94:54976) 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 -- 172.21.15.94:0/970296152 learned_addr learned my addr 172.21.15.94:0/970296152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 -- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f035815eb20 msgr2=0x7f035815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f035815eb20 0x7f035815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 -- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03580b5650 msgr2=0x7f03581422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03580b5650 0x7f03581422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.526+0000 7f034f400640 1 -- 172.21.15.94:0/970296152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03580ccbc0 con 0x7f03580d2b20 2024-05-27T12:34:51.527 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f034f400640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f034000d800 tx=0x7f034000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.528 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000b6c0 con 0x7f03580d2b20 2024-05-27T12:34:51.529 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0340019070 con 0x7f03580d2b20 2024-05-27T12:34:51.529 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03580ccea0 con 0x7f03580d2b20 2024-05-27T12:34:51.529 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000bac0 con 0x7f03580d2b20 2024-05-27T12:34:51.529 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.527+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03580cd460 con 0x7f03580d2b20 2024-05-27T12:34:51.530 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.528+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035804fce0 con 0x7f03580d2b20 2024-05-27T12:34:51.530 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.529+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f034000bc60 con 0x7f03580d2b20 2024-05-27T12:34:51.531 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.530+0000 7f0354c00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 0x7f033406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.531 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.530+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f034008b250 con 0x7f03580d2b20 2024-05-27T12:34:51.531 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.530+0000 7f0357e00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 0x7f033406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.532 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.532+0000 7f0357e00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 0x7f033406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03480012e0 tx=0x7f0348007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.537 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.535+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f034005a450 con 0x7f03580d2b20 2024-05-27T12:34:51.747 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.744+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f035806dad0 con 0x7f03580d2b20 2024-05-27T12:34:51.747 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.746+0000 7f0354c00640 1 -- 172.21.15.94:0/970296152 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f0340059d50 con 0x7f03580d2b20 2024-05-27T12:34:51.748 INFO:teuthology.misc.health.smithi094.stdout: 2024-05-27T12:34:51.748 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":13,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:51.647356+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 msgr2=0x7f033406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 0x7f033406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03480012e0 tx=0x7f0348007d50 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 msgr2=0x7f035815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f034000d800 tx=0x7f034000dcc0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 shutdown_connections 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f033406c860 0x7f033406ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f035815eb20 0x7f035815ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03580d2b20 0x7f035815a800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 --2- 172.21.15.94:0/970296152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03580b5650 0x7f03581422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 >> 172.21.15.94:0/970296152 conn(0x7f03580b4350 msgr2=0x7f0358159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:51.749 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.748+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 shutdown_connections 2024-05-27T12:34:51.750 INFO:teuthology.misc.health.smithi094.stderr:2024-05-27T12:34:51.749+0000 7f035ee00640 1 -- 172.21.15.94:0/970296152 wait complete. 2024-05-27T12:34:51.761 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-27T12:34:51.761 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-27T12:34:51.761 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T12:34:51.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T12:34:51.768 INFO:tasks.ceph.mds.a:Started 2024-05-27T12:34:51.768 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T12:34:51.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T12:34:51.776 INFO:tasks.ceph.mds.c:Started 2024-05-27T12:34:51.776 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-27T12:34:51.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-27T12:34:51.784 INFO:tasks.ceph.mds.b:Started 2024-05-27T12:34:51.784 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-27T12:34:51.784 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-27T12:34:51.791 INFO:tasks.ceph.mds.d:Started 2024-05-27T12:34:51.791 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-27T12:34:51.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:34:51.903 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-05-27T12:34:51.907 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-05-27T12:34:51.935 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2024-05-27T12:34:51.936 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2024-05-27T12:34:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.956+0000 7fb866200640 1 Processor -- start 2024-05-27T12:34:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.956+0000 7fb866200640 1 -- start start 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb866200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8600cf400 0x7fb8600cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb866200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb8600c5710 0x7fb8600c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb866200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86006f450 con 0x7fb86015a250 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb866200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb860160f70 con 0x7fb8600c5710 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb866200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb8600cfd80 con 0x7fb8600cf400 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb8600c5710 0x7fb8600c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb8600c5710 0x7fb8600c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55002/0 (socket says 172.21.15.94:55002) 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 -- 172.21.15.94:0/148920877 learned_addr learned my addr 172.21.15.94:0/148920877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85ea00640 1 --2- 172.21.15.94:0/148920877 >> v2:172.21.15.97:3300/0 conn(0x7fb8600cf400 0x7fb8600cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 -- 172.21.15.94:0/148920877 >> v2:172.21.15.97:3300/0 conn(0x7fb8600cf400 msgr2=0x7fb8600cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 --2- 172.21.15.94:0/148920877 >> v2:172.21.15.97:3300/0 conn(0x7fb8600cf400 0x7fb8600cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 -- 172.21.15.94:0/148920877 >> v1:172.21.15.97:6789/0 conn(0x7fb86015a250 legacy=0x7fb86006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.958+0000 7fb85fe00640 1 -- 172.21.15.94:0/148920877 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb860160360 con 0x7fb8600c5710 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.959+0000 7fb85fe00640 1 --2- 172.21.15.94:0/148920877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600c5710 0x7fb8600c5b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b2ad0 tx=0x7fb85802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf310b9b5d5d857 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.959+0000 7fb85e000640 1 -- 172.21.15.94:0/148920877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858036070 con 0x7fb8600c5710 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb85e000640 1 -- 172.21.15.94:0/148920877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb858031600 con 0x7fb8600c5710 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb85e000640 1 -- 172.21.15.94:0/148920877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580319d0 con 0x7fb8600c5710 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 -- 172.21.15.94:0/148920877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600c5710 msgr2=0x7fb8600c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 --2- 172.21.15.94:0/148920877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600c5710 0x7fb8600c5b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b2ad0 tx=0x7fb85802a6f0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 -- 172.21.15.94:0/148920877 shutdown_connections 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 --2- 172.21.15.94:0/148920877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600c5710 0x7fb8600c5b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 --2- 172.21.15.94:0/148920877 >> v2:172.21.15.97:3300/0 conn(0x7fb8600cf400 0x7fb8600cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 -- 172.21.15.94:0/148920877 >> 172.21.15.94:0/148920877 conn(0x7fb8600b3830 msgr2=0x7fb8600b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 -- 172.21.15.94:0/148920877 shutdown_connections 2024-05-27T12:34:51.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.960+0000 7fb866200640 1 -- 172.21.15.94:0/148920877 wait complete. 2024-05-27T12:34:51.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.962+0000 7fb866200640 1 Processor -- start 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.962+0000 7fb866200640 1 -- start start 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8600c5710 0x7fb86014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86015a250 0x7fb8600b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb860160c00 con 0x7fb8600cf400 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb860160920 con 0x7fb86015a250 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb866200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb86015a630 con 0x7fb8600c5710 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85f400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8600c5710 0x7fb86014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55014/0 (socket says 172.21.15.94:55014) 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 -- 172.21.15.94:0/1661177661 learned_addr learned my addr 172.21.15.94:0/1661177661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:51.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85fe00640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86015a250 0x7fb8600b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 -- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8600c5710 msgr2=0x7fb86014e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8600c5710 0x7fb86014e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 -- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86015a250 msgr2=0x7fb8600b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86015a250 0x7fb8600b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.963+0000 7fb85ea00640 1 -- 172.21.15.94:0/1661177661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86014f420 con 0x7fb8600cf400 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb85ea00640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d800 tx=0x7fb84c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00b6c0 con 0x7fb8600cf400 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c019070 con 0x7fb8600cf400 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86014f6e0 con 0x7fb8600cf400 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00ba80 con 0x7fb8600cf400 2024-05-27T12:34:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.964+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86014ff80 con 0x7fb8600cf400 2024-05-27T12:34:51.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.965+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86014fad0 con 0x7fb8600cf400 2024-05-27T12:34:51.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.966+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fb84c00bc20 con 0x7fb8600cf400 2024-05-27T12:34:51.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.967+0000 7fb857400640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 0x7fb84006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:51.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.967+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fb84c08b9c0 con 0x7fb8600cf400 2024-05-27T12:34:51.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.967+0000 7fb85f400640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 0x7fb84006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:51.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.968+0000 7fb85f400640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 0x7fb84006ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb850006fd0 tx=0x7fb850009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:51.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:51.971+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb84c059cb0 con 0x7fb8600cf400 2024-05-27T12:34:52.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.161+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb86014f8b0 con 0x7fb8600cf400 2024-05-27T12:34:52.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.163+0000 7fb857400640 1 -- 172.21.15.94:0/1661177661 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7fb84c05dfb0 con 0x7fb8600cf400 2024-05-27T12:34:52.162 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:52.163 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:51.647356+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 msgr2=0x7fb84006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 0x7fb84006ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb850006fd0 tx=0x7fb850009060 comp rx=0 tx=0).stop 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 msgr2=0x7fb86014d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d800 tx=0x7fb84c00dcc0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 shutdown_connections 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb84006c860 0x7fb84006ed20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86015a250 0x7fb8600b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8600cf400 0x7fb86014d2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 --2- 172.21.15.94:0/1661177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8600c5710 0x7fb86014e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.165+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 >> 172.21.15.94:0/1661177661 conn(0x7fb8600b3830 msgr2=0x7fb86015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.166+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 shutdown_connections 2024-05-27T12:34:52.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.166+0000 7fb866200640 1 -- 172.21.15.94:0/1661177661 wait complete. 2024-05-27T12:34:52.181 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-27T12:34:52.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:34:52.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.364+0000 7f9517e00640 1 Processor -- start 2024-05-27T12:34:52.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.364+0000 7f9517e00640 1 -- start start 2024-05-27T12:34:52.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.365+0000 7f9517e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f95180b3460 0x7f95180b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.365+0000 7f9517e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f95180afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.365+0000 7f9517e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95180aeb20 con 0x7f9518144de0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.365+0000 7f9517e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f95180af130 con 0x7f95180b3460 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.365+0000 7f9517e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f95181451c0 con 0x7f95180af5c0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516a00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9518144de0 0x7f951806ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46382/0 (socket says 172.21.15.94:46382) 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516a00640 1 -- 172.21.15.94:0/536785520 learned_addr learned my addr 172.21.15.94:0/536785520 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516000640 1 --2- 172.21.15.94:0/536785520 >> v2:172.21.15.94:3300/0 conn(0x7f95180b3460 0x7f95180b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9515600640 1 -- 172.21.15.94:0/536785520 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886207854 0 0) 0x7f95180aeb20 con 0x7f9518144de0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9515600640 1 -- 172.21.15.94:0/536785520 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94fc003520 con 0x7f9518144de0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9517400640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f95180afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516000640 1 -- 172.21.15.94:0/536785520 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 msgr2=0x7f95180afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516000640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f95180afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516000640 1 -- 172.21.15.94:0/536785520 >> v1:172.21.15.94:6789/0 conn(0x7f9518144de0 legacy=0x7f951806ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9516000640 1 -- 172.21.15.94:0/536785520 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951806e880 con 0x7f95180b3460 2024-05-27T12:34:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.366+0000 7f9517400640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f95180afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.367+0000 7f9516000640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f95180b38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95000097a0 tx=0x7f950002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e5c794698e0d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.367+0000 7f9515600640 1 -- 172.21.15.94:0/536785520 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500037070 con 0x7f95180b3460 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.367+0000 7f9515600640 1 -- 172.21.15.94:0/536785520 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9500032440 con 0x7f95180b3460 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.367+0000 7f9515600640 1 -- 172.21.15.94:0/536785520 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500032810 con 0x7f95180b3460 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 -- 172.21.15.94:0/536785520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 msgr2=0x7f95180b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f95180b38f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95000097a0 tx=0x7f950002a180 comp rx=0 tx=0).stop 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 -- 172.21.15.94:0/536785520 shutdown_connections 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f95180afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 --2- 172.21.15.94:0/536785520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f95180b38f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 -- 172.21.15.94:0/536785520 >> 172.21.15.94:0/536785520 conn(0x7f95180b2160 msgr2=0x7f95180b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:52.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 -- 172.21.15.94:0/536785520 shutdown_connections 2024-05-27T12:34:52.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.368+0000 7f9517e00640 1 -- 172.21.15.94:0/536785520 wait complete. 2024-05-27T12:34:52.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.370+0000 7f9517e00640 1 Processor -- start 2024-05-27T12:34:52.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.370+0000 7f9517e00640 1 -- start start 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f951813ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518144de0 0x7f9518147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f951806f3c0 con 0x7f95180b3460 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95180aedc0 con 0x7f9518144de0 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f951806e4d0 con 0x7f95180af5c0 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55044/0 (socket says 172.21.15.94:55044) 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 -- 172.21.15.94:0/2744522349 learned_addr learned my addr 172.21.15.94:0/2744522349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516a00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f951813ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9517400640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518144de0 0x7f9518147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 -- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 msgr2=0x7f951813ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f951813ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 -- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518144de0 msgr2=0x7f9518147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518144de0 0x7f9518147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.371+0000 7f9516000640 1 -- 172.21.15.94:0/2744522349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518145c40 con 0x7f95180b3460 2024-05-27T12:34:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f9516000640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9500035040 tx=0x7f95000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500037040 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9500030500 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500030910 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9518145f20 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.372+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181464e0 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.373+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9518075460 con 0x7f95180b3460 2024-05-27T12:34:52.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.374+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9500044020 con 0x7f95180b3460 2024-05-27T12:34:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.375+0000 7f950ea00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 0x7f94f406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.375+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f95000ab880 con 0x7f95180b3460 2024-05-27T12:34:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.376+0000 7f9516a00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 0x7f94f406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:52.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.377+0000 7f9516a00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 0x7f94f406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9508000f60 tx=0x7f9508009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.379+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f950007a250 con 0x7f95180b3460 2024-05-27T12:34:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.599+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95181609d0 con 0x7f95180b3460 2024-05-27T12:34:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.600+0000 7f950ea00640 1 -- 172.21.15.94:0/2744522349 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f9500079b50 con 0x7f95180b3460 2024-05-27T12:34:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:52.600 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T12:34:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-05-27T12:34:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.602+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 msgr2=0x7f94f406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.602+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 0x7f94f406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9508000f60 tx=0x7f9508009040 comp rx=0 tx=0).stop 2024-05-27T12:34:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.602+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 msgr2=0x7f9518145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.602+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9500035040 tx=0x7f95000046c0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 shutdown_connections 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f406c860 0x7f94f406ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518144de0 0x7f9518147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95180b3460 0x7f9518145340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 --2- 172.21.15.94:0/2744522349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f95180af5c0 0x7f951813ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 >> 172.21.15.94:0/2744522349 conn(0x7f95180b2160 msgr2=0x7f9518144110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 shutdown_connections 2024-05-27T12:34:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:52.603+0000 7f9517e00640 1 -- 172.21.15.94:0/2744522349 wait complete. 2024-05-27T12:34:54.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:34:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.821+0000 7fe59f200640 1 Processor -- start 2024-05-27T12:34:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.822+0000 7fe59f200640 1 -- start start 2024-05-27T12:34:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe5a00d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5a00b4510 0x7fe5a00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59f200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a015ee10 con 0x7fe5a015ea30 2024-05-27T12:34:54.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59f200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a00d2830 con 0x7fe5a00b4510 2024-05-27T12:34:54.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a00d2a60 con 0x7fe5a00d4ae0 2024-05-27T12:34:54.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59de00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe5a015ea30 0x7fe5a00d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46396/0 (socket says 172.21.15.94:46396) 2024-05-27T12:34:54.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.823+0000 7fe59de00640 1 -- 172.21.15.94:0/2707610943 learned_addr learned my addr 172.21.15.94:0/2707610943 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:54.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59d400640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe5a00d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59e800640 1 --2- 172.21.15.94:0/2707610943 >> v2:172.21.15.97:3300/0 conn(0x7fe5a00b4510 0x7fe5a00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285090180 0 0) 0x7fe5a015ee10 con 0x7fe5a015ea30 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe584003520 con 0x7fe5a015ea30 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4164732463 0 0) 0x7fe584003520 con 0x7fe5a015ea30 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 msgr2=0x7fe5a00d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe5a00d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 >> v2:172.21.15.97:3300/0 conn(0x7fe5a00b4510 msgr2=0x7fe5a00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 --2- 172.21.15.94:0/2707610943 >> v2:172.21.15.97:3300/0 conn(0x7fe5a00b4510 0x7fe5a00cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a00d2d40 con 0x7fe5a015ea30 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fe590001670 con 0x7fe5a015ea30 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.824+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 >> v1:172.21.15.94:6789/0 conn(0x7fe5a015ea30 legacy=0x7fe5a00d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:54.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840039b0 0x7fe584003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe584004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe584004d40 0x7fe5840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00d2a60 con 0x7fe584004d40 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe58400b9c0 con 0x7fe5840039b0 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe58400bbc0 con 0x7fe5a00d4ae0 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe584004d40 0x7fe5840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59de00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840039b0 0x7fe584003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59d400640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe584004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 -- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 msgr2=0x7fe584004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:54.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe584004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 -- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840039b0 msgr2=0x7fe584003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840039b0 0x7fe584003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59e800640 1 -- 172.21.15.94:0/2707610943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584003520 con 0x7fe584004d40 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59d400640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5a00d4ae0 0x7fe584004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.825+0000 7fe59de00640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840039b0 0x7fe584003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.826+0000 7fe59e800640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe584004d40 0x7fe5840051c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe598009fc0 tx=0x7fe584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10da27e28178c1e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598009c10 con 0x7fe584004d40 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe598036070 con 0x7fe584004d40 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59ca00640 1 -- 172.21.15.94:0/2707610943 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59802d070 con 0x7fe584004d40 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59f200640 1 -- 172.21.15.94:0/2707610943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe584004d40 msgr2=0x7fe5840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59f200640 1 --2- 172.21.15.94:0/2707610943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe584004d40 0x7fe5840051c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe598009fc0 tx=0x7fe584005830 comp rx=0 tx=0).stop 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59e800640 1 -- 172.21.15.94:0/2707610943 reap_dead start 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59f200640 1 -- 172.21.15.94:0/2707610943 shutdown_connections 2024-05-27T12:34:54.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.827+0000 7fe59f200640 1 -- 172.21.15.94:0/2707610943 >> 172.21.15.94:0/2707610943 conn(0x7fe5a009da70 msgr2=0x7fe5a008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.828+0000 7fe59f200640 1 -- 172.21.15.94:0/2707610943 shutdown_connections 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.828+0000 7fe59f200640 1 -- 172.21.15.94:0/2707610943 wait complete. 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.830+0000 7fe59f200640 1 Processor -- start 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.830+0000 7fe59f200640 1 -- start start 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe584004d40 0x7fe5a00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a00c9c00 0x7fe5a015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00d35e0 con 0x7fe5a015b720 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00d3300 con 0x7fe5a00c9c00 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a00d22f0 con 0x7fe584004d40 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55050/0 (socket says 172.21.15.94:55050) 2024-05-27T12:34:54.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59e800640 1 -- 172.21.15.94:0/2638460202 learned_addr learned my addr 172.21.15.94:0/2638460202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:54.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59de00640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe584004d40 0x7fe5a00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.831+0000 7fe59d400640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a00c9c00 0x7fe5a015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 -- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe584004d40 msgr2=0x7fe5a00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe584004d40 0x7fe5a00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 -- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a00c9c00 msgr2=0x7fe5a015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a00c9c00 0x7fe5a015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 -- 172.21.15.94:0/2638460202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a00cd590 con 0x7fe5a015b720 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe59e800640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe59802af10 tx=0x7fe598009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:54.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.832+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598009c10 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.833+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe598003990 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.833+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598003d60 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.833+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a00ca150 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.833+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a00ca710 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.834+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a00c8250 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.836+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe598005d10 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.836+0000 7fe596000640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 0x7fe58006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.837+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7fe5980acaf0 con 0x7fe5a015b720 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.837+0000 7fe59de00640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 0x7fe58006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:54.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.838+0000 7fe59de00640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 0x7fe58006ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5900017e0 tx=0x7fe590009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:54.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:54.842+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe59807b2b0 con 0x7fe5a015b720 2024-05-27T12:34:55.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.097+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5a00cfb00 con 0x7fe5a015b720 2024-05-27T12:34:55.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.097+0000 7fe596000640 1 -- 172.21.15.94:0/2638460202 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7fe59807abb0 con 0x7fe5a015b720 2024-05-27T12:34:55.097 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:55.097 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T12:34:55.097 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-05-27T12:34:55.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 msgr2=0x7fe58006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 0x7fe58006ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5900017e0 tx=0x7fe590009040 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 msgr2=0x7fe5a0161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe59802af10 tx=0x7fe598009750 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 shutdown_connections 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe58006c860 0x7fe58006ed20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a00c9c00 0x7fe5a015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5a015b720 0x7fe5a0161360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 --2- 172.21.15.94:0/2638460202 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe584004d40 0x7fe5a00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 >> 172.21.15.94:0/2638460202 conn(0x7fe5a009da70 msgr2=0x7fe5a00b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 shutdown_connections 2024-05-27T12:34:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.100+0000 7fe59f200640 1 -- 172.21.15.94:0/2638460202 wait complete. 2024-05-27T12:34:55.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:34:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.298+0000 7ff101800640 1 Processor -- start 2024-05-27T12:34:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.299+0000 7ff101800640 1 -- start start 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff101800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff0fc145cf0 0x7ff0fc160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff101800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc0ceae0 con 0x7ff0fc1460d0 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff101800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc1602c0 con 0x7ff0fc071d60 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff101800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff0fc1605a0 con 0x7ff0fc145cf0 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff0fbe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff0fc145cf0 0x7ff0fc160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff0f3400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff0fc1460d0 0x7ff0fc0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46404/0 (socket says 172.21.15.94:46404) 2024-05-27T12:34:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.300+0000 7ff0f3400640 1 -- 172.21.15.94:0/3405063978 learned_addr learned my addr 172.21.15.94:0/3405063978 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.301+0000 7ff0fbe00640 1 -- 172.21.15.94:0/3405063978 >> v1:172.21.15.97:6789/0 conn(0x7ff0fc071d60 legacy=0x7ff0fc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.301+0000 7ff0fbe00640 1 -- 172.21.15.94:0/3405063978 >> v1:172.21.15.94:6789/0 conn(0x7ff0fc1460d0 legacy=0x7ff0fc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.301+0000 7ff0fbe00640 1 -- 172.21.15.94:0/3405063978 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc160880 con 0x7ff0fc145cf0 2024-05-27T12:34:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.302+0000 7ff0fbe00640 1 --2- 172.21.15.94:0/3405063978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc160da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009d90 tx=0x7ff0ec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3882f9195b9007f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.302+0000 7ff0fb400640 1 -- 172.21.15.94:0/3405063978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec007740 con 0x7ff0fc145cf0 2024-05-27T12:34:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.302+0000 7ff0fb400640 1 -- 172.21.15.94:0/3405063978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0078e0 con 0x7ff0fc145cf0 2024-05-27T12:34:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.302+0000 7ff0fb400640 1 -- 172.21.15.94:0/3405063978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec007cb0 con 0x7ff0fc145cf0 2024-05-27T12:34:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 -- 172.21.15.94:0/3405063978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc145cf0 msgr2=0x7ff0fc160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 --2- 172.21.15.94:0/3405063978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc160da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009d90 tx=0x7ff0ec02a6f0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 -- 172.21.15.94:0/3405063978 shutdown_connections 2024-05-27T12:34:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 --2- 172.21.15.94:0/3405063978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc160da0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 -- 172.21.15.94:0/3405063978 >> 172.21.15.94:0/3405063978 conn(0x7ff0fc0b2160 msgr2=0x7ff0fc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 -- 172.21.15.94:0/3405063978 shutdown_connections 2024-05-27T12:34:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.303+0000 7ff101800640 1 -- 172.21.15.94:0/3405063978 wait complete. 2024-05-27T12:34:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.305+0000 7ff101800640 1 Processor -- start 2024-05-27T12:34:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.305+0000 7ff101800640 1 -- start start 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc1460d0 0x7ff0fc13da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc06eb00 con 0x7ff0fc071d60 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc0cedc0 con 0x7ff0fc145cf0 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff101800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc06ec80 con 0x7ff0fc1460d0 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff0fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff0fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55074/0 (socket says 172.21.15.94:55074) 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.306+0000 7ff0fbe00640 1 -- 172.21.15.94:0/1460677336 learned_addr learned my addr 172.21.15.94:0/1460677336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0f3400640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff100e00640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc1460d0 0x7ff0fc13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 -- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc1460d0 msgr2=0x7ff0fc13da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc1460d0 0x7ff0fc13da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 -- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc145cf0 msgr2=0x7ff0fc141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 -- 172.21.15.94:0/1460677336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc141d90 con 0x7ff0fc071d60 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0fbe00640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009d90 tx=0x7ff0ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec040070 con 0x7ff0fc071d60 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec031c40 con 0x7ff0fc071d60 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec045460 con 0x7ff0fc071d60 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.307+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc142070 con 0x7ff0fc071d60 2024-05-27T12:34:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.308+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc142630 con 0x7ff0fc071d60 2024-05-27T12:34:55.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.309+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc075460 con 0x7ff0fc071d60 2024-05-27T12:34:55.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.310+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0ec039040 con 0x7ff0fc071d60 2024-05-27T12:34:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.310+0000 7ff0f8c00640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.311+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff0ec0ad240 con 0x7ff0fc071d60 2024-05-27T12:34:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.311+0000 7ff0f3400640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.312+0000 7ff0f3400640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 0x7ff0d806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005880 tx=0x7ff0e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.315+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0ec07ba40 con 0x7ff0fc071d60 2024-05-27T12:34:55.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.528+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0fc072e00 con 0x7ff0fc071d60 2024-05-27T12:34:55.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.529+0000 7ff0f8c00640 1 -- 172.21.15.94:0/1460677336 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7ff0ec07b340 con 0x7ff0fc071d60 2024-05-27T12:34:55.529 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:55.529 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:55.173187+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 msgr2=0x7ff0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 0x7ff0d806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005880 tx=0x7ff0e4009040 comp rx=0 tx=0).stop 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 msgr2=0x7ff0fc142e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009d90 tx=0x7ff0ec004a40 comp rx=0 tx=0).stop 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 shutdown_connections 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff0d806c8f0 0x7ff0d806edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc145cf0 0x7ff0fc141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc1460d0 0x7ff0fc13da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 --2- 172.21.15.94:0/1460677336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0fc071d60 0x7ff0fc142e30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.532+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 >> 172.21.15.94:0/1460677336 conn(0x7ff0fc0b2160 msgr2=0x7ff0fc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.533+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 shutdown_connections 2024-05-27T12:34:55.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.533+0000 7ff101800640 1 -- 172.21.15.94:0/1460677336 wait complete. 2024-05-27T12:34:55.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:34:55.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:34:55.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.758+0000 7f1187e00640 1 Processor -- start 2024-05-27T12:34:55.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.758+0000 7f1187e00640 1 -- start start 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f118815e900 0x7f1188067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11880ba3f0 0x7f11880b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11880b66d0 0x7f1188162480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f11880b6b40 con 0x7f118815e900 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f11880b7840 con 0x7f11880ba3f0 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1187e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1188067f30 con 0x7f11880b66d0 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1186a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f118815e900 0x7f1188067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1186a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f118815e900 0x7f1188067630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55090/0 (socket says 172.21.15.94:55090) 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.760+0000 7f1186a00640 1 -- 172.21.15.94:0/2259993344 learned_addr learned my addr 172.21.15.94:0/2259993344 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186000640 1 --2- 172.21.15.94:0/2259993344 >> v2:172.21.15.97:3300/0 conn(0x7f11880ba3f0 0x7f11880b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1187400640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11880b66d0 0x7f1188162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186a00640 1 -- 172.21.15.94:0/2259993344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11880b66d0 msgr2=0x7f1188162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186a00640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11880b66d0 0x7f1188162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186a00640 1 -- 172.21.15.94:0/2259993344 >> v2:172.21.15.97:3300/0 conn(0x7f11880ba3f0 msgr2=0x7f11880b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186a00640 1 --2- 172.21.15.94:0/2259993344 >> v2:172.21.15.97:3300/0 conn(0x7f11880ba3f0 0x7f11880b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.761+0000 7f1186a00640 1 -- 172.21.15.94:0/2259993344 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1188068210 con 0x7f118815e900 2024-05-27T12:34:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.762+0000 7f1186a00640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118815e900 0x7f1188067630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f11880d29c0 tx=0x7f117802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9077726a6215f204 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1185600640 1 -- 172.21.15.94:0/2259993344 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178037070 con 0x7f118815e900 2024-05-27T12:34:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1185600640 1 -- 172.21.15.94:0/2259993344 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1178032440 con 0x7f118815e900 2024-05-27T12:34:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1185600640 1 -- 172.21.15.94:0/2259993344 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178032810 con 0x7f118815e900 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 -- 172.21.15.94:0/2259993344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118815e900 msgr2=0x7f1188067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118815e900 0x7f1188067630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f11880d29c0 tx=0x7f117802a270 comp rx=0 tx=0).stop 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 -- 172.21.15.94:0/2259993344 shutdown_connections 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11880b66d0 0x7f1188162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 --2- 172.21.15.94:0/2259993344 >> v2:172.21.15.97:3300/0 conn(0x7f11880ba3f0 0x7f11880b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 --2- 172.21.15.94:0/2259993344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118815e900 0x7f1188067630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 -- 172.21.15.94:0/2259993344 >> 172.21.15.94:0/2259993344 conn(0x7f11880d1190 msgr2=0x7f11880d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.763+0000 7f1187e00640 1 -- 172.21.15.94:0/2259993344 shutdown_connections 2024-05-27T12:34:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.764+0000 7f1187e00640 1 -- 172.21.15.94:0/2259993344 wait complete. 2024-05-27T12:34:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.765+0000 7f1187e00640 1 Processor -- start 2024-05-27T12:34:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.765+0000 7f1187e00640 1 -- start start 2024-05-27T12:34:55.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11880b66d0 0x7f11881553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f118815e900 0x7f118815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11880d2df0 con 0x7f11880ba3f0 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11880d2b10 con 0x7f11880b66d0 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.766+0000 7f1187e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f118815ece0 con 0x7f118815e900 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55094/0 (socket says 172.21.15.94:55094) 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 -- 172.21.15.94:0/2830638210 learned_addr learned my addr 172.21.15.94:0/2830638210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186a00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11880b66d0 0x7f11881553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1187400640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f118815e900 0x7f118815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 -- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f118815e900 msgr2=0x7f118815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f118815e900 0x7f118815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 -- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11880b66d0 msgr2=0x7f11881553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11880b66d0 0x7f11881553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 -- 172.21.15.94:0/2830638210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1188155af0 con 0x7f11880ba3f0 2024-05-27T12:34:55.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.767+0000 7f1186000640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f117000efc0 tx=0x7f117000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.768+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170018020 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.768+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1170014480 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.768+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170014850 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.768+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1188155dd0 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.768+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1188156390 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.769+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1188075460 con 0x7f11880ba3f0 2024-05-27T12:34:55.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.770+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1170005d10 con 0x7f11880ba3f0 2024-05-27T12:34:55.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.771+0000 7f117f400640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 0x7f116406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:55.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.771+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f117008c460 con 0x7f11880ba3f0 2024-05-27T12:34:55.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.771+0000 7f1186a00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 0x7f116406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:55.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.772+0000 7f1186a00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 0x7f116406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1178034000 tx=0x7f117802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.776+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1170022030 con 0x7f11880ba3f0 2024-05-27T12:34:55.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.978+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f11881621f0 con 0x7f11880ba3f0 2024-05-27T12:34:55.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.979+0000 7f117f400640 1 -- 172.21.15.94:0/2830638210 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f1170010040 con 0x7f11880ba3f0 2024-05-27T12:34:55.978 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 msgr2=0x7f116406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 0x7f116406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1178034000 tx=0x7f117802e000 comp rx=0 tx=0).stop 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 msgr2=0x7f1188148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f117000efc0 tx=0x7f117000c490 comp rx=0 tx=0).stop 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 shutdown_connections 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f116406c8f0 0x7f116406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f118815e900 0x7f118815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11880ba3f0 0x7f1188148a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 --2- 172.21.15.94:0/2830638210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11880b66d0 0x7f11881553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:55.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.981+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 >> 172.21.15.94:0/2830638210 conn(0x7f11880d1190 msgr2=0x7f1188160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:55.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.982+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 shutdown_connections 2024-05-27T12:34:55.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:55.982+0000 7f1187e00640 1 -- 172.21.15.94:0/2830638210 wait complete. 2024-05-27T12:34:55.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-27T12:34:56.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.200+0000 7f0728e00640 1 Processor -- start 2024-05-27T12:34:56.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.201+0000 7f0728e00640 1 -- start start 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.202+0000 7f0728e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072415eaf0 0x7f07240d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.202+0000 7f0728e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07240ba5e0 0x7f07240d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.202+0000 7f0728e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07240d3280 con 0x7f07240d4320 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.202+0000 7f0728e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f07240b89d0 con 0x7f072415eaf0 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.202+0000 7f0728e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f07240d3e90 con 0x7f07240ba5e0 2024-05-27T12:34:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.203+0000 7f0721e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072415eaf0 0x7f07240d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.203+0000 7f0721400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07240ba5e0 0x7f07240d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.203+0000 7f0721e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072415eaf0 0x7f07240d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:46918/0 (socket says 172.21.15.94:46918) 2024-05-27T12:34:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.203+0000 7f0721e00640 1 -- 172.21.15.94:0/401834904 learned_addr learned my addr 172.21.15.94:0/401834904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.204+0000 7f0721e00640 1 -- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07240ba5e0 msgr2=0x7f07240d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.204+0000 7f0721e00640 1 --2- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07240ba5e0 0x7f07240d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.204+0000 7f0721e00640 1 -- 172.21.15.94:0/401834904 >> v1:172.21.15.97:6789/0 conn(0x7f07240d4320 legacy=0x7f07240b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.204+0000 7f0721e00640 1 -- 172.21.15.94:0/401834904 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07240d3560 con 0x7f072415eaf0 2024-05-27T12:34:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.205+0000 7f0721e00640 1 --2- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072415eaf0 0x7f07240d2910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0714009e50 tx=0x7f071402a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90256324cc8cb58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0720a00640 1 -- 172.21.15.94:0/401834904 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714036070 con 0x7f072415eaf0 2024-05-27T12:34:56.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0720a00640 1 -- 172.21.15.94:0/401834904 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0714031600 con 0x7f072415eaf0 2024-05-27T12:34:56.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0720a00640 1 -- 172.21.15.94:0/401834904 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140319d0 con 0x7f072415eaf0 2024-05-27T12:34:56.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 -- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072415eaf0 msgr2=0x7f07240d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 --2- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072415eaf0 0x7f07240d2910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0714009e50 tx=0x7f071402a7a0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 -- 172.21.15.94:0/401834904 shutdown_connections 2024-05-27T12:34:56.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 --2- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07240ba5e0 0x7f07240d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 --2- 172.21.15.94:0/401834904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072415eaf0 0x7f07240d2910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.206+0000 7f0728e00640 1 -- 172.21.15.94:0/401834904 >> 172.21.15.94:0/401834904 conn(0x7f072406cdb0 msgr2=0x7f072406d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.207+0000 7f0728e00640 1 -- 172.21.15.94:0/401834904 shutdown_connections 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.207+0000 7f0728e00640 1 -- 172.21.15.94:0/401834904 wait complete. 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.208+0000 7f0728e00640 1 Processor -- start 2024-05-27T12:34:56.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.208+0000 7f0728e00640 1 -- start start 2024-05-27T12:34:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07240d4320 0x7f07240cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f072415eaf0 0x7f07240ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f072415eed0 con 0x7f07240ba5e0 2024-05-27T12:34:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07240d3b20 con 0x7f07240d4320 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.209+0000 7f0728e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f072415f050 con 0x7f072415eaf0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55110/0 (socket says 172.21.15.94:55110) 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 -- 172.21.15.94:0/3280615717 learned_addr learned my addr 172.21.15.94:0/3280615717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721400640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07240d4320 0x7f07240cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0722800640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f072415eaf0 0x7f07240ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 -- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f072415eaf0 msgr2=0x7f07240ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f072415eaf0 0x7f07240ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 -- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07240d4320 msgr2=0x7f07240cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07240d4320 0x7f07240cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 -- 172.21.15.94:0/3280615717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07240abf90 con 0x7f07240ba5e0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.210+0000 7f0721e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0714034040 tx=0x7f0714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.211+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714036050 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.211+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0714042070 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.211+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714009510 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.211+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07240ac270 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.211+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07240ac830 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.212+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0724075460 con 0x7f07240ba5e0 2024-05-27T12:34:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.213+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0714007c90 con 0x7f07240ba5e0 2024-05-27T12:34:56.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.214+0000 7f071a000640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 0x7f070006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.214+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f07140ac670 con 0x7f07240ba5e0 2024-05-27T12:34:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.215+0000 7f0721400640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 0x7f070006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.216+0000 7f0721400640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 0x7f070006ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07240cf8b0 tx=0x7f070c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.219+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f071407adf0 con 0x7f07240ba5e0 2024-05-27T12:34:56.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.426+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f072406bd30 con 0x7f07240ba5e0 2024-05-27T12:34:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.427+0000 7f071a000640 1 -- 172.21.15.94:0/3280615717 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f071407a6f0 con 0x7f07240ba5e0 2024-05-27T12:34:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:34:56.426 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:34:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:34:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-05-27T12:34:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-05-27T12:34:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:34:56.427 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:34:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 msgr2=0x7f070006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 0x7f070006ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07240cf8b0 tx=0x7f070c008040 comp rx=0 tx=0).stop 2024-05-27T12:34:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 msgr2=0x7f072415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0714034040 tx=0x7f0714004a40 comp rx=0 tx=0).stop 2024-05-27T12:34:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 shutdown_connections 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f070006c920 0x7f070006ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07240d4320 0x7f07240cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f072415eaf0 0x7f07240ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 --2- 172.21.15.94:0/3280615717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07240ba5e0 0x7f072415ad00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.429+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 >> 172.21.15.94:0/3280615717 conn(0x7f072406cdb0 msgr2=0x7f072415d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.430+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 shutdown_connections 2024-05-27T12:34:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.430+0000 7f0728e00640 1 -- 172.21.15.94:0/3280615717 wait complete. 2024-05-27T12:34:56.447 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T12:34:56.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T12:34:56.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.652+0000 7fdcb2c00640 1 Processor -- start 2024-05-27T12:34:56.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.652+0000 7fdcb2c00640 1 -- start start 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.653+0000 7fdcb2c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcac15bdc0 0x7fdcac071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcb2c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcac0b4620 0x7fdcac06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcb2c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcac0727b0 con 0x7fdcac0723a0 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcb2c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fdcac06e1d0 con 0x7fdcac15bdc0 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcb2c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdcac0b4be0 con 0x7fdcac0b4620 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcac15bdc0 0x7fdcac071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fdcac15bdc0 0x7fdcac071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55126/0 (socket says 172.21.15.94:55126) 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 -- 172.21.15.94:0/3134069768 learned_addr learned my addr 172.21.15.94:0/3134069768 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:56.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcb0c00640 1 --2- 172.21.15.94:0/3134069768 >> v2:172.21.15.97:3300/0 conn(0x7fdcac0b4620 0x7fdcac06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 -- 172.21.15.94:0/3134069768 >> v2:172.21.15.97:3300/0 conn(0x7fdcac0b4620 msgr2=0x7fdcac06e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:56.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 --2- 172.21.15.94:0/3134069768 >> v2:172.21.15.97:3300/0 conn(0x7fdcac0b4620 0x7fdcac06e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 -- 172.21.15.94:0/3134069768 >> v1:172.21.15.97:6789/0 conn(0x7fdcac0723a0 legacy=0x7fdcac0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:56.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.654+0000 7fdcabe00640 1 -- 172.21.15.94:0/3134069768 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac0b4ea0 con 0x7fdcac15bdc0 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.655+0000 7fdcabe00640 1 --2- 172.21.15.94:0/3134069768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac15bdc0 0x7fdcac071e60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdcac06e140 tx=0x7fdc9c02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d192e2e53e946e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.656+0000 7fdcab400640 1 -- 172.21.15.94:0/3134069768 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c03b070 con 0x7fdcac15bdc0 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.656+0000 7fdcab400640 1 -- 172.21.15.94:0/3134069768 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c0336a0 con 0x7fdcac15bdc0 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.656+0000 7fdcab400640 1 -- 172.21.15.94:0/3134069768 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c032050 con 0x7fdcac15bdc0 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 -- 172.21.15.94:0/3134069768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac15bdc0 msgr2=0x7fdcac071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/3134069768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac15bdc0 0x7fdcac071e60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdcac06e140 tx=0x7fdc9c02c870 comp rx=0 tx=0).stop 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 -- 172.21.15.94:0/3134069768 shutdown_connections 2024-05-27T12:34:56.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/3134069768 >> v2:172.21.15.97:3300/0 conn(0x7fdcac0b4620 0x7fdcac06e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/3134069768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac15bdc0 0x7fdcac071e60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 -- 172.21.15.94:0/3134069768 >> 172.21.15.94:0/3134069768 conn(0x7fdcac0b2240 msgr2=0x7fdcac0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:56.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 -- 172.21.15.94:0/3134069768 shutdown_connections 2024-05-27T12:34:56.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.657+0000 7fdcb2c00640 1 -- 172.21.15.94:0/3134069768 wait complete. 2024-05-27T12:34:56.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.659+0000 7fdcb2c00640 1 Processor -- start 2024-05-27T12:34:56.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.659+0000 7fdcb2c00640 1 -- start start 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcac0723a0 0x7fdcac1e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15bdc0 0x7fdcac1e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac15c1a0 con 0x7fdcac0b4620 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06de60 con 0x7fdcac15bdc0 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb2c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac15c320 con 0x7fdcac0723a0 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55138/0 (socket says 172.21.15.94:55138) 2024-05-27T12:34:56.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 -- 172.21.15.94:0/2786834857 learned_addr learned my addr 172.21.15.94:0/2786834857 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:56.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcb0c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15bdc0 0x7fdcac1e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdcabe00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcac0723a0 0x7fdcac1e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 -- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcac0723a0 msgr2=0x7fdcac1e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcac0723a0 0x7fdcac1e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 -- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15bdc0 msgr2=0x7fdcac1e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15bdc0 0x7fdcac1e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.660+0000 7fdca3400640 1 -- 172.21.15.94:0/2786834857 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac0c6150 con 0x7fdcac0b4620 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdca3400640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400b3f0 tx=0x7fdc9400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94015020 con 0x7fdcac0b4620 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc94011830 con 0x7fdcac0b4620 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011c20 con 0x7fdcac0b4620 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac0c6430 con 0x7fdcac0b4620 2024-05-27T12:34:56.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.661+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac1e9340 con 0x7fdcac0b4620 2024-05-27T12:34:56.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.662+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac068a70 con 0x7fdcac0b4620 2024-05-27T12:34:56.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.663+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc94018020 con 0x7fdcac0b4620 2024-05-27T12:34:56.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.664+0000 7fdca8c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 0x7fdc8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:56.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.664+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdc9408b390 con 0x7fdcac0b4620 2024-05-27T12:34:56.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.665+0000 7fdcabe00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 0x7fdc8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:56.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.666+0000 7fdcabe00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 0x7fdc8806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c038000 tx=0x7fdc9c031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:56.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.669+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc9401f030 con 0x7fdcac0b4620 2024-05-27T12:34:56.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:56.871+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdcac0d1240 con 0x7fdcac0b4620 2024-05-27T12:34:57.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.873+0000 7fdca8c00640 1 -- 172.21.15.94:0/2786834857 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fdc940172c0 con 0x7fdcac0b4620 2024-05-27T12:34:57.873 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-05-27T12:34:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.876+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 msgr2=0x7fdc8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.876+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 0x7fdc8806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c038000 tx=0x7fdc9c031000 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 msgr2=0x7fdcac1544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400b3f0 tx=0x7fdc9400b8b0 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 shutdown_connections 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdc8806c8f0 0x7fdc8806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15bdc0 0x7fdcac1e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac0b4620 0x7fdcac1544f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 --2- 172.21.15.94:0/2786834857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcac0723a0 0x7fdcac1e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 >> 172.21.15.94:0/2786834857 conn(0x7fdcac0b2240 msgr2=0x7fdcac06f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 shutdown_connections 2024-05-27T12:34:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:57.877+0000 7fdcb2c00640 1 -- 172.21.15.94:0/2786834857 wait complete. 2024-05-27T12:34:57.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T12:34:58.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.095+0000 7f858b000640 1 Processor -- start 2024-05-27T12:34:58.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.095+0000 7f858b000640 1 -- start start 2024-05-27T12:34:58.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f858b000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f85840b3aa0 0x7f85840d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f858b000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85840b3e80 con 0x7f85840d4fb0 2024-05-27T12:34:58.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f858b000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85840d3440 con 0x7f858406caa0 2024-05-27T12:34:58.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f858b000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f85840d2020 con 0x7f85840b3aa0 2024-05-27T12:34:58.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f8589200640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f85840d4fb0 0x7f85840d1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46416/0 (socket says 172.21.15.94:46416) 2024-05-27T12:34:58.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f8589200640 1 -- 172.21.15.94:0/354711062 learned_addr learned my addr 172.21.15.94:0/354711062 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:58.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f8589c00640 1 --2- 172.21.15.94:0/354711062 >> v2:172.21.15.97:3300/0 conn(0x7f85840b3aa0 0x7f85840d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.097+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604133795 0 0) 0x7f85840b3e80 con 0x7f85840d4fb0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f856c003520 con 0x7f85840d4fb0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 855184637 0 0) 0x7f856c003520 con 0x7f85840d4fb0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 >> v2:172.21.15.97:3300/0 conn(0x7f85840b3aa0 msgr2=0x7f85840d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 --2- 172.21.15.94:0/354711062 >> v2:172.21.15.97:3300/0 conn(0x7f85840b3aa0 0x7f85840d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 >> v1:172.21.15.97:6789/0 conn(0x7f858406caa0 legacy=0x7f85840cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858406e210 con 0x7f85840d4fb0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f8570005090 con 0x7f85840d4fb0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 >> v1:172.21.15.94:6789/0 conn(0x7f85840d4fb0 legacy=0x7f85840d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 0x7f856c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c004800 0x7f856c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f856c0051c0 0x7f856c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85840d2020 con 0x7f856c0039b0 2024-05-27T12:34:58.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f856c00c110 con 0x7f856c004800 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f856c00c310 con 0x7f856c0051c0 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.098+0000 7f8589c00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 0x7f856c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589200640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c004800 0x7f856c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589c00640 1 -- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f856c0051c0 msgr2=0x7f856c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589c00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f856c0051c0 0x7f856c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589c00640 1 -- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c004800 msgr2=0x7f856c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589c00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c004800 0x7f856c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f8589c00640 1 -- 172.21.15.94:0/354711062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c003520 con 0x7f856c0039b0 2024-05-27T12:34:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.099+0000 7f858a600640 1 -- 172.21.15.94:0/354711062 reap_dead start 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.100+0000 7f8589c00640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 0x7f856c003dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f857800ba30 tx=0x7f856c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82cca620886a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.100+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578030070 con 0x7f856c0039b0 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.100+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857802ade0 con 0x7f856c0039b0 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.100+0000 7f8583e00640 1 -- 172.21.15.94:0/354711062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857800b600 con 0x7f856c0039b0 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 -- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 msgr2=0x7f856c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 0x7f856c003dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f857800ba30 tx=0x7f856c005830 comp rx=0 tx=0).stop 2024-05-27T12:34:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 -- 172.21.15.94:0/354711062 shutdown_connections 2024-05-27T12:34:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 --2- 172.21.15.94:0/354711062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f856c0039b0 0x7f856c003dd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 -- 172.21.15.94:0/354711062 >> 172.21.15.94:0/354711062 conn(0x7f858408bfc0 msgr2=0x7f85840d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 -- 172.21.15.94:0/354711062 shutdown_connections 2024-05-27T12:34:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.101+0000 7f858b000640 1 -- 172.21.15.94:0/354711062 wait complete. 2024-05-27T12:34:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.103+0000 7f858b000640 1 Processor -- start 2024-05-27T12:34:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.103+0000 7f858b000640 1 -- start start 2024-05-27T12:34:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 0x7f85841f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f85840cd6d0 0x7f85840cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85840d1110 con 0x7f85840c95e0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f858406e7d0 con 0x7f856c0039b0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858b000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f85840d13f0 con 0x7f85840cd6d0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55158/0 (socket says 172.21.15.94:55158) 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 -- 172.21.15.94:0/3255031598 learned_addr learned my addr 172.21.15.94:0/3255031598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f8589c00640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 0x7f85841f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f8589200640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f85840cd6d0 0x7f85840cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 -- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f85840cd6d0 msgr2=0x7f85840cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f85840cd6d0 0x7f85840cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 -- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 msgr2=0x7f85841f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 0x7f85841f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f858a600640 1 -- 172.21.15.94:0/3255031598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85841f8200 con 0x7f85840c95e0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.104+0000 7f8589c00640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 0x7f85841f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f858a600640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85840cf290 tx=0x7f857c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c017070 con 0x7f85840c95e0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c005d60 con 0x7f85840c95e0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c00a850 con 0x7f85840c95e0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85840ca310 con 0x7f85840c95e0 2024-05-27T12:34:58.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.105+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85840ca8d0 con 0x7f85840c95e0 2024-05-27T12:34:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.106+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85840cb2b0 con 0x7f85840c95e0 2024-05-27T12:34:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.108+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f857c01a070 con 0x7f85840c95e0 2024-05-27T12:34:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.108+0000 7f8581600640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 0x7f856406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.109+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f857c08ab10 con 0x7f85840c95e0 2024-05-27T12:34:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.109+0000 7f8589c00640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 0x7f856406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.110+0000 7f8589c00640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 0x7f856406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f857800ba00 tx=0x7f8578030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:58.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.114+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85840cb2b0 con 0x7f85840c95e0 2024-05-27T12:34:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.315+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8584075460 con 0x7f85840c95e0 2024-05-27T12:34:58.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:34:58.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:34:58.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.879+0000 7f8581600640 1 -- 172.21.15.94:0/3255031598 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f857c059110 con 0x7f85840c95e0 2024-05-27T12:34:58.879 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 msgr2=0x7f856406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 0x7f856406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f857800ba00 tx=0x7f8578030000 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 msgr2=0x7f85840c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85840cf290 tx=0x7f857c00cb90 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 shutdown_connections 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f856406c8f0 0x7f856406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85840c95e0 0x7f85840c9a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f85840cd6d0 0x7f85840cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 --2- 172.21.15.94:0/3255031598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f856c0039b0 0x7f85841f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:58.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 >> 172.21.15.94:0/3255031598 conn(0x7f858408bfc0 msgr2=0x7f85841613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:58.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 shutdown_connections 2024-05-27T12:34:58.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:58.883+0000 7f858b000640 1 -- 172.21.15.94:0/3255031598 wait complete. 2024-05-27T12:34:58.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T12:34:59.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.089+0000 7efc93e00640 1 Processor -- start 2024-05-27T12:34:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.089+0000 7efc93e00640 1 -- start start 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.090+0000 7efc93e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efc94161d00 0x7efc940d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.090+0000 7efc93e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940d30b0 con 0x7efc940b61a0 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.090+0000 7efc93e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940d3390 con 0x7efc940ba470 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.090+0000 7efc93e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7efc941620e0 con 0x7efc94161d00 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc92a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7efc94161d00 0x7efc940d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc93400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7efc940b61a0 0x7efc940b65a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:46426/0 (socket says 172.21.15.94:46426) 2024-05-27T12:34:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc93400640 1 -- 172.21.15.94:0/2131335060 learned_addr learned my addr 172.21.15.94:0/2131335060 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:34:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc92a00640 1 -- 172.21.15.94:0/2131335060 >> v1:172.21.15.97:6789/0 conn(0x7efc940ba470 legacy=0x7efc940d2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:34:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc92a00640 1 -- 172.21.15.94:0/2131335060 >> v1:172.21.15.94:6789/0 conn(0x7efc940b61a0 legacy=0x7efc940b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc92a00640 1 -- 172.21.15.94:0/2131335060 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc940d3670 con 0x7efc94161d00 2024-05-27T12:34:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.091+0000 7efc91600640 1 -- 172.21.15.94:0/2131335060 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517850912 0 0) 0x7efc940d30b0 con 0x7efc940b61a0 2024-05-27T12:34:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.092+0000 7efc92a00640 1 --2- 172.21.15.94:0/2131335060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc940d4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc840097a0 tx=0x7efc8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accbedbd4d9d6fb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.092+0000 7efc91600640 1 -- 172.21.15.94:0/2131335060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84037070 con 0x7efc94161d00 2024-05-27T12:34:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc91600640 1 -- 172.21.15.94:0/2131335060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84032440 con 0x7efc94161d00 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc91600640 1 -- 172.21.15.94:0/2131335060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84032810 con 0x7efc94161d00 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 -- 172.21.15.94:0/2131335060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 msgr2=0x7efc940d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 --2- 172.21.15.94:0/2131335060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc940d4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc840097a0 tx=0x7efc8402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 -- 172.21.15.94:0/2131335060 shutdown_connections 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 --2- 172.21.15.94:0/2131335060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc940d4150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 -- 172.21.15.94:0/2131335060 >> 172.21.15.94:0/2131335060 conn(0x7efc9406d180 msgr2=0x7efc940d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 -- 172.21.15.94:0/2131335060 shutdown_connections 2024-05-27T12:34:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.093+0000 7efc93e00640 1 -- 172.21.15.94:0/2131335060 wait complete. 2024-05-27T12:34:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.095+0000 7efc93e00640 1 Processor -- start 2024-05-27T12:34:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.095+0000 7efc93e00640 1 -- start start 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efc940b61a0 0x7efc940cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 0x7efc94151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc94169330 con 0x7efc94161d00 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc940d2e50 con 0x7efc940ba470 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7efc941694b0 con 0x7efc940b61a0 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55184/0 (socket says 172.21.15.94:55184) 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.096+0000 7efc93400640 1 -- 172.21.15.94:0/297203080 learned_addr learned my addr 172.21.15.94:0/297203080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 -- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efc940b61a0 msgr2=0x7efc940cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efc940b61a0 0x7efc940cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc92000640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 0x7efc94151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 -- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 msgr2=0x7efc94151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 0x7efc94151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 -- 172.21.15.94:0/297203080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc940c9560 con 0x7efc94161d00 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc92000640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 0x7efc94151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93400640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c7a0 tx=0x7efc8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c018070 con 0x7efc94161d00 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8c00a640 con 0x7efc94161d00 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc940cd750 con 0x7efc94161d00 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00aab0 con 0x7efc94161d00 2024-05-27T12:34:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.097+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc940cdd10 con 0x7efc94161d00 2024-05-27T12:34:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.098+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94075460 con 0x7efc94161d00 2024-05-27T12:34:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.099+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc8c01b070 con 0x7efc94161d00 2024-05-27T12:34:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.100+0000 7efc8aa00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 0x7efc7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:34:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.100+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7efc8c08b150 con 0x7efc94161d00 2024-05-27T12:34:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.101+0000 7efc92a00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 0x7efc7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:34:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.102+0000 7efc92a00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 0x7efc7406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc84034000 tx=0x7efc8402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:34:59.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.105+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc8c059540 con 0x7efc94161d00 2024-05-27T12:34:59.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.346+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efc9406bba0 con 0x7efc94161d00 2024-05-27T12:34:59.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.930+0000 7efc8aa00640 1 -- 172.21.15.94:0/297203080 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7efc8c058e40 con 0x7efc94161d00 2024-05-27T12:34:59.932 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T12:34:59.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.938+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 msgr2=0x7efc7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.938+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 0x7efc7406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc84034000 tx=0x7efc8402c000 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.938+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 msgr2=0x7efc94152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.938+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c7a0 tx=0x7efc8c00cc60 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 shutdown_connections 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efc7406c8f0 0x7efc7406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc940ba470 0x7efc94151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc94161d00 0x7efc94152490 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 --2- 172.21.15.94:0/297203080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efc940b61a0 0x7efc940cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.939+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 >> 172.21.15.94:0/297203080 conn(0x7efc9406d180 msgr2=0x7efc940b9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.940+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 shutdown_connections 2024-05-27T12:34:59.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:34:59.940+0000 7efc93e00640 1 -- 172.21.15.94:0/297203080 wait complete. 2024-05-27T12:34:59.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:00.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.147+0000 7f4ae1c00640 1 Processor -- start 2024-05-27T12:35:00.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.147+0000 7f4ae1c00640 1 -- start start 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.148+0000 7f4ae1c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4adc15e920 0x7f4adc162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.148+0000 7f4ae1c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4adc0b2f10 0x7f4adc071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4ae1c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4adc15ed00 con 0x7f4adc072460 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4ae1c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4adc072890 con 0x7f4adc15e920 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4ae1c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f4adc0cd580 con 0x7f4adc0b2f10 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adaa00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4adc15e920 0x7f4adc162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adb400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4adc072460 0x7f4adc06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48026/0 (socket says 172.21.15.94:48026) 2024-05-27T12:35:00.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adb400640 1 -- 172.21.15.94:0/1088312072 learned_addr learned my addr 172.21.15.94:0/1088312072 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4ada000640 1 --2- 172.21.15.94:0/1088312072 >> v2:172.21.15.97:3300/0 conn(0x7f4adc0b2f10 0x7f4adc071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adaa00640 1 -- 172.21.15.94:0/1088312072 >> v2:172.21.15.97:3300/0 conn(0x7f4adc0b2f10 msgr2=0x7f4adc071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adaa00640 1 --2- 172.21.15.94:0/1088312072 >> v2:172.21.15.97:3300/0 conn(0x7f4adc0b2f10 0x7f4adc071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adaa00640 1 -- 172.21.15.94:0/1088312072 >> v1:172.21.15.94:6789/0 conn(0x7f4adc072460 legacy=0x7f4adc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4adaa00640 1 -- 172.21.15.94:0/1088312072 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc0cd860 con 0x7f4adc15e920 2024-05-27T12:35:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.149+0000 7f4ad9600640 1 -- 172.21.15.94:0/1088312072 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416641748 0 0) 0x7f4adc15ed00 con 0x7f4adc072460 2024-05-27T12:35:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.150+0000 7f4adaa00640 1 --2- 172.21.15.94:0/1088312072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc15e920 0x7f4adc162e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0ca110 tx=0x7f4acc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dc54409ad12b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ad9600640 1 -- 172.21.15.94:0/1088312072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc037070 con 0x7f4adc15e920 2024-05-27T12:35:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ad9600640 1 -- 172.21.15.94:0/1088312072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc032440 con 0x7f4adc15e920 2024-05-27T12:35:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ad9600640 1 -- 172.21.15.94:0/1088312072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc032810 con 0x7f4adc15e920 2024-05-27T12:35:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 -- 172.21.15.94:0/1088312072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc15e920 msgr2=0x7f4adc162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/1088312072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc15e920 0x7f4adc162e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0ca110 tx=0x7f4acc02a1a0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 -- 172.21.15.94:0/1088312072 shutdown_connections 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/1088312072 >> v2:172.21.15.97:3300/0 conn(0x7f4adc0b2f10 0x7f4adc071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/1088312072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc15e920 0x7f4adc162e40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.151+0000 7f4ae1c00640 1 -- 172.21.15.94:0/1088312072 >> 172.21.15.94:0/1088312072 conn(0x7f4adc0b3830 msgr2=0x7f4adc0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.152+0000 7f4ae1c00640 1 -- 172.21.15.94:0/1088312072 shutdown_connections 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.152+0000 7f4ae1c00640 1 -- 172.21.15.94:0/1088312072 wait complete. 2024-05-27T12:35:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.153+0000 7f4ae1c00640 1 Processor -- start 2024-05-27T12:35:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.153+0000 7f4ae1c00640 1 -- start start 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4adc0b2f10 0x7f4adc0b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 0x7f4adc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc0ce100 con 0x7f4adc072460 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc0cde20 con 0x7f4adc0b2f10 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.154+0000 7f4ae1c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4adc163380 con 0x7f4adc15e920 2024-05-27T12:35:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56302/0 (socket says 172.21.15.94:56302) 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 -- 172.21.15.94:0/598444144 learned_addr learned my addr 172.21.15.94:0/598444144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4ada000640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4adc0b2f10 0x7f4adc0b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adb400640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 0x7f4adc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 -- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 msgr2=0x7f4adc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 0x7f4adc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 -- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4adc0b2f10 msgr2=0x7f4adc0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4adc0b2f10 0x7f4adc0b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 -- 172.21.15.94:0/598444144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc145ff0 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adb400640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 0x7f4adc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.155+0000 7f4adaa00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009890 tx=0x7f4acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.156+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc037040 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.156+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc042070 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.156+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc030bd0 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.156+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4adc1462d0 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.156+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4adc146890 con 0x7f4adc072460 2024-05-27T12:35:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.157+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4adc075460 con 0x7f4adc072460 2024-05-27T12:35:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.158+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4acc047460 con 0x7f4adc072460 2024-05-27T12:35:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.159+0000 7f4ad2a00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.159+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4acc0ac7d0 con 0x7f4adc072460 2024-05-27T12:35:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.160+0000 7f4ada000640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.161+0000 7f4ada000640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 0x7f4abc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006fd0 tx=0x7f4ac8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.164+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4acc032e20 con 0x7f4adc072460 2024-05-27T12:35:00.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.365+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4adc0730f0 con 0x7f4adc072460 2024-05-27T12:35:00.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.368+0000 7f4ad2a00640 1 -- 172.21.15.94:0/598444144 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f4acc045360 con 0x7f4adc072460 2024-05-27T12:35:00.368 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:00.368 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:59.347730+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 msgr2=0x7f4abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 0x7f4abc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006fd0 tx=0x7f4ac8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 msgr2=0x7f4adc15a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009890 tx=0x7f4acc0046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 shutdown_connections 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4adc0b2f10 0x7f4adc0b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4adc15e920 0x7f4adc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 --2- 172.21.15.94:0/598444144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4adc072460 0x7f4adc15a8b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.370+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 >> 172.21.15.94:0/598444144 conn(0x7f4adc0b3830 msgr2=0x7f4adc0cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:00.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.371+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 shutdown_connections 2024-05-27T12:35:00.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.371+0000 7f4ae1c00640 1 -- 172.21.15.94:0/598444144 wait complete. 2024-05-27T12:35:00.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:00.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.580+0000 7fc27be00640 1 Processor -- start 2024-05-27T12:35:00.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.580+0000 7fc27be00640 1 -- start start 2024-05-27T12:35:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.582+0000 7fc27be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2740cf400 0x7fc2740cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.582+0000 7fc27be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 0x7fc2740c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.582+0000 7fc27be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27406f450 con 0x7fc27415a250 2024-05-27T12:35:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.582+0000 7fc27be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc274160f70 con 0x7fc2740cf400 2024-05-27T12:35:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.582+0000 7fc27be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc2740cfd80 con 0x7fc2740c5710 2024-05-27T12:35:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.583+0000 7fc278a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2740cf400 0x7fc2740cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.583+0000 7fc279e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 0x7fc2740c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.583+0000 7fc278a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2740cf400 0x7fc2740cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:58876/0 (socket says 172.21.15.94:58876) 2024-05-27T12:35:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.583+0000 7fc278a00640 1 -- 172.21.15.94:0/1889598829 learned_addr learned my addr 172.21.15.94:0/1889598829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.584+0000 7fc278a00640 1 -- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 msgr2=0x7fc2740c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.584+0000 7fc278a00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 0x7fc2740c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.584+0000 7fc278a00640 1 -- 172.21.15.94:0/1889598829 >> v1:172.21.15.97:6789/0 conn(0x7fc27415a250 legacy=0x7fc27406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.584+0000 7fc278a00640 1 -- 172.21.15.94:0/1889598829 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc274160360 con 0x7fc2740cf400 2024-05-27T12:35:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.584+0000 7fc279e00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 0x7fc2740c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:35:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.585+0000 7fc278a00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740cf400 0x7fc2740cf840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc260009e50 tx=0x7fc26002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae07f1a75ef7509f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.585+0000 7fc273e00640 1 -- 172.21.15.94:0/1889598829 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260036070 con 0x7fc2740cf400 2024-05-27T12:35:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.585+0000 7fc273e00640 1 -- 172.21.15.94:0/1889598829 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260031600 con 0x7fc2740cf400 2024-05-27T12:35:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.585+0000 7fc273e00640 1 -- 172.21.15.94:0/1889598829 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2600319d0 con 0x7fc2740cf400 2024-05-27T12:35:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 -- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740cf400 msgr2=0x7fc2740cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740cf400 0x7fc2740cf840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc260009e50 tx=0x7fc26002a710 comp rx=0 tx=0).stop 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 -- 172.21.15.94:0/1889598829 shutdown_connections 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc2740c5710 0x7fc2740c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 --2- 172.21.15.94:0/1889598829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740cf400 0x7fc2740cf840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 -- 172.21.15.94:0/1889598829 >> 172.21.15.94:0/1889598829 conn(0x7fc2740b3830 msgr2=0x7fc2740b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 -- 172.21.15.94:0/1889598829 shutdown_connections 2024-05-27T12:35:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.586+0000 7fc27be00640 1 -- 172.21.15.94:0/1889598829 wait complete. 2024-05-27T12:35:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.588+0000 7fc27be00640 1 Processor -- start 2024-05-27T12:35:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.588+0000 7fc27be00640 1 -- start start 2024-05-27T12:35:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 0x7fc274156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc27415a250 0x7fc2741f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc274160c00 con 0x7fc2740cf400 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc274160920 con 0x7fc2740c5710 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc27be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc27415a630 con 0x7fc27415a250 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc278a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc278a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56308/0 (socket says 172.21.15.94:56308) 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.589+0000 7fc278a00640 1 -- 172.21.15.94:0/3734659897 learned_addr learned my addr 172.21.15.94:0/3734659897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc279400640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 0x7fc274156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc279e00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc27415a250 0x7fc2741f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 -- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc27415a250 msgr2=0x7fc2741f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc27415a250 0x7fc2741f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 -- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 msgr2=0x7fc274156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 0x7fc274156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 -- 172.21.15.94:0/3734659897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc274158ef0 con 0x7fc2740cf400 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc279400640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 0x7fc274156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.590+0000 7fc278a00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc260034040 tx=0x7fc260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.591+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260036050 con 0x7fc2740cf400 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.591+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260042070 con 0x7fc2740cf400 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.591+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260009510 con 0x7fc2740cf400 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.591+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2741591d0 con 0x7fc2740cf400 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.591+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc274159790 con 0x7fc2740cf400 2024-05-27T12:35:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.592+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2740cfff0 con 0x7fc2740cf400 2024-05-27T12:35:00.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.593+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc260007c70 con 0x7fc2740cf400 2024-05-27T12:35:00.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.594+0000 7fc271600640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:00.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.594+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fc2600aca20 con 0x7fc2740cf400 2024-05-27T12:35:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.595+0000 7fc279400640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:00.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.596+0000 7fc279400640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 0x7fc25406ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:00.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.598+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc26007ad20 con 0x7fc2740cf400 2024-05-27T12:35:00.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.789+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc274075460 con 0x7fc2740cf400 2024-05-27T12:35:00.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.791+0000 7fc271600640 1 -- 172.21.15.94:0/3734659897 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7fc26007a620 con 0x7fc2740cf400 2024-05-27T12:35:00.791 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:00.791 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:34:59.347730+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:00.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 msgr2=0x7fc25406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 0x7fc25406ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 msgr2=0x7fc274158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc260034040 tx=0x7fc260004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 shutdown_connections 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc27415a250 0x7fc2741f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2740cf400 0x7fc274158630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 --2- 172.21.15.94:0/3734659897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2740c5710 0x7fc274156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 >> 172.21.15.94:0/3734659897 conn(0x7fc2740b3830 msgr2=0x7fc27415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 shutdown_connections 2024-05-27T12:35:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:00.794+0000 7fc27be00640 1 -- 172.21.15.94:0/3734659897 wait complete. 2024-05-27T12:35:00.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-27T12:35:01.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.017+0000 7fe7bfa00640 1 Processor -- start 2024-05-27T12:35:01.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.018+0000 7fe7bfa00640 1 -- start start 2024-05-27T12:35:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bfa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b80b46d0 0x7fe7b80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bfa00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe7b8162f20 0x7fe7b806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bfa00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b8166c80 con 0x7fe7b8162b40 2024-05-27T12:35:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bfa00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b8167800 con 0x7fe7b8162f20 2024-05-27T12:35:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bfa00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b8166f60 con 0x7fe7b80b46d0 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bd000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe7b8162b40 0x7fe7b80b3fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48032/0 (socket says 172.21.15.94:48032) 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bd000640 1 -- 172.21.15.94:0/3937678080 learned_addr learned my addr 172.21.15.94:0/3937678080 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.019+0000 7fe7bda00640 1 --2- 172.21.15.94:0/3937678080 >> v2:172.21.15.94:3300/0 conn(0x7fe7b8162f20 0x7fe7b806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7afe00640 1 --2- 172.21.15.94:0/3937678080 >> v2:172.21.15.97:3300/0 conn(0x7fe7b80b46d0 0x7fe7b80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7b7e00640 1 -- 172.21.15.94:0/3937678080 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202113137 0 0) 0x7fe7b8166c80 con 0x7fe7b8162b40 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7b7e00640 1 -- 172.21.15.94:0/3937678080 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe798003520 con 0x7fe7b8162b40 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7bda00640 1 -- 172.21.15.94:0/3937678080 >> v2:172.21.15.97:3300/0 conn(0x7fe7b80b46d0 msgr2=0x7fe7b80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7bda00640 1 --2- 172.21.15.94:0/3937678080 >> v2:172.21.15.97:3300/0 conn(0x7fe7b80b46d0 0x7fe7b80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7bda00640 1 -- 172.21.15.94:0/3937678080 >> v1:172.21.15.94:6789/0 conn(0x7fe7b8162b40 legacy=0x7fe7b80b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.020+0000 7fe7bda00640 1 -- 172.21.15.94:0/3937678080 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b8167240 con 0x7fe7b8162f20 2024-05-27T12:35:01.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.021+0000 7fe7bda00640 1 --2- 172.21.15.94:0/3937678080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8162f20 0x7fe7b806ee70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806c670 tx=0x7fe7b002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594de775adf7ac50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:01.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.021+0000 7fe7b7e00640 1 -- 172.21.15.94:0/3937678080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0007740 con 0x7fe7b8162f20 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.021+0000 7fe7b7e00640 1 -- 172.21.15.94:0/3937678080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b00078e0 con 0x7fe7b8162f20 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.021+0000 7fe7b7e00640 1 -- 172.21.15.94:0/3937678080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0007cb0 con 0x7fe7b8162f20 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 -- 172.21.15.94:0/3937678080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8162f20 msgr2=0x7fe7b806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/3937678080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8162f20 0x7fe7b806ee70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7b806c670 tx=0x7fe7b002a5f0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 -- 172.21.15.94:0/3937678080 shutdown_connections 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/3937678080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8162f20 0x7fe7b806ee70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/3937678080 >> v2:172.21.15.97:3300/0 conn(0x7fe7b80b46d0 0x7fe7b80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 -- 172.21.15.94:0/3937678080 >> 172.21.15.94:0/3937678080 conn(0x7fe7b80b2240 msgr2=0x7fe7b80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 -- 172.21.15.94:0/3937678080 shutdown_connections 2024-05-27T12:35:01.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.022+0000 7fe7bfa00640 1 -- 172.21.15.94:0/3937678080 wait complete. 2024-05-27T12:35:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.023+0000 7fe7bfa00640 1 Processor -- start 2024-05-27T12:35:01.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.024+0000 7fe7bfa00640 1 -- start start 2024-05-27T12:35:01.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.024+0000 7fe7bfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.024+0000 7fe7bfa00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b8162b40 0x7fe7b8165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.024+0000 7fe7bfa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b8162f20 0x7fe7b8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.024+0000 7fe7bfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8166a40 con 0x7fe7b80b46d0 2024-05-27T12:35:01.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bfa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81668c0 con 0x7fe7b8162f20 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bfa00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b80afce0 con 0x7fe7b8162b40 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56332/0 (socket says 172.21.15.94:56332) 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 -- 172.21.15.94:0/2682281539 learned_addr learned my addr 172.21.15.94:0/2682281539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7afe00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b8162b40 0x7fe7b8165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 -- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b8162b40 msgr2=0x7fe7b8165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b8162b40 0x7fe7b8165140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 -- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b8162f20 msgr2=0x7fe7b8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bda00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b8162f20 0x7fe7b8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b8162f20 0x7fe7b8154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 -- 172.21.15.94:0/2682281539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80cd6c0 con 0x7fe7b80b46d0 2024-05-27T12:35:01.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.025+0000 7fe7bd000640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c430 tx=0x7fe7a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.026+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.026+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a0005ce0 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.026+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00072f0 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.026+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b80cd9a0 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.026+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b80cdf60 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.027+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8068a70 con 0x7fe7b80b46d0 2024-05-27T12:35:01.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.028+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7a001a070 con 0x7fe7b80b46d0 2024-05-27T12:35:01.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.029+0000 7fe7b5600640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 0x7fe79406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:01.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.029+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7a008afe0 con 0x7fe7b80b46d0 2024-05-27T12:35:01.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.029+0000 7fe7afe00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 0x7fe79406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:01.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.030+0000 7fe7afe00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 0x7fe79406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7b815ddc0 tx=0x7fe7a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:01.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.033+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7a0059440 con 0x7fe7b80b46d0 2024-05-27T12:35:01.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.271+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe7b8157570 con 0x7fe7b80b46d0 2024-05-27T12:35:01.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.950+0000 7fe7b5600640 1 -- 172.21.15.94:0/2682281539 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fe7a0058d40 con 0x7fe7b80b46d0 2024-05-27T12:35:01.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 msgr2=0x7fe79406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 0x7fe79406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7b815ddc0 tx=0x7fe7a8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:01.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 msgr2=0x7fe7b8153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c430 tx=0x7fe7a000c8f0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 shutdown_connections 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79406c8f0 0x7fe79406edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b8162f20 0x7fe7b8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b8162b40 0x7fe7b8165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 --2- 172.21.15.94:0/2682281539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b80b46d0 0x7fe7b8153b80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.953+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 >> 172.21.15.94:0/2682281539 conn(0x7fe7b80b2240 msgr2=0x7fe7b8074bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.954+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 shutdown_connections 2024-05-27T12:35:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:01.954+0000 7fe7bfa00640 1 -- 172.21.15.94:0/2682281539 wait complete. 2024-05-27T12:35:01.975 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-27T12:35:01.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-27T12:35:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.180+0000 7fb425e00640 1 Processor -- start 2024-05-27T12:35:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.180+0000 7fb425e00640 1 -- start start 2024-05-27T12:35:02.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb425e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 0x7fb4200cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb425e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4200d1f40 0x7fb4200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb425e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42015eee0 con 0x7fb4200d3e50 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb425e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb4200d5700 con 0x7fb4200d1f40 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb425e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb4200d4660 con 0x7fb42015eb00 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb41f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4200d1f40 0x7fb4200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb41f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb4200d1f40 0x7fb4200b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56334/0 (socket says 172.21.15.94:56334) 2024-05-27T12:35:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb41f400640 1 -- 172.21.15.94:0/2019094904 learned_addr learned my addr 172.21.15.94:0/2019094904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.182+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 0x7fb4200cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.183+0000 7fb41f400640 1 -- 172.21.15.94:0/2019094904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 msgr2=0x7fb4200cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.183+0000 7fb41f400640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 0x7fb4200cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.183+0000 7fb41f400640 1 -- 172.21.15.94:0/2019094904 >> v1:172.21.15.97:6789/0 conn(0x7fb4200d3e50 legacy=0x7fb4200d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.183+0000 7fb41f400640 1 -- 172.21.15.94:0/2019094904 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4200d4940 con 0x7fb4200d1f40 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.183+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 0x7fb4200cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.184+0000 7fb41f400640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4200d1f40 0x7fb4200b78f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb41802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580420430db4fd3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:02.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.184+0000 7fb41d600640 1 -- 172.21.15.94:0/2019094904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418037070 con 0x7fb4200d1f40 2024-05-27T12:35:02.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.184+0000 7fb41d600640 1 -- 172.21.15.94:0/2019094904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb418032440 con 0x7fb4200d1f40 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.184+0000 7fb41d600640 1 -- 172.21.15.94:0/2019094904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418032810 con 0x7fb4200d1f40 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 -- 172.21.15.94:0/2019094904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4200d1f40 msgr2=0x7fb4200b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4200d1f40 0x7fb4200b78f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb41802a290 comp rx=0 tx=0).stop 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 -- 172.21.15.94:0/2019094904 shutdown_connections 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4200d1f40 0x7fb4200b78f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 --2- 172.21.15.94:0/2019094904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb42015eb00 0x7fb4200cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 -- 172.21.15.94:0/2019094904 >> 172.21.15.94:0/2019094904 conn(0x7fb420067a50 msgr2=0x7fb420067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 -- 172.21.15.94:0/2019094904 shutdown_connections 2024-05-27T12:35:02.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.185+0000 7fb425e00640 1 -- 172.21.15.94:0/2019094904 wait complete. 2024-05-27T12:35:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.186+0000 7fb425e00640 1 Processor -- start 2024-05-27T12:35:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.187+0000 7fb425e00640 1 -- start start 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.187+0000 7fb425e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 0x7fb4200b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.187+0000 7fb425e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4200d3e50 0x7fb4201601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb425e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb425e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4200d51e0 con 0x7fb42015eb00 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb425e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4200d4f00 con 0x7fb4200d3e50 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb425e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb4200d4290 con 0x7fb4200d1f40 2024-05-27T12:35:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56340/0 (socket says 172.21.15.94:56340) 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 -- 172.21.15.94:0/2172253903 learned_addr learned my addr 172.21.15.94:0/2172253903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 0x7fb4200b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41e000640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4200d3e50 0x7fb4201601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 -- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 msgr2=0x7fb4200b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 0x7fb4200b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 -- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4200d3e50 msgr2=0x7fb4201601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4200d3e50 0x7fb4201601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 -- 172.21.15.94:0/2172253903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4200b3600 con 0x7fb42015eb00 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 0x7fb4200b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.188+0000 7fb41f400640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.189+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418037040 con 0x7fb42015eb00 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.189+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb418042070 con 0x7fb42015eb00 2024-05-27T12:35:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.189+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4201606f0 con 0x7fb42015eb00 2024-05-27T12:35:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.189+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418030bd0 con 0x7fb42015eb00 2024-05-27T12:35:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.189+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4200c93e0 con 0x7fb42015eb00 2024-05-27T12:35:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.190+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420068a70 con 0x7fb42015eb00 2024-05-27T12:35:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.191+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb418047460 con 0x7fb42015eb00 2024-05-27T12:35:02.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.192+0000 7fb417400640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 0x7fb3fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:02.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.192+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4180ac780 con 0x7fb42015eb00 2024-05-27T12:35:02.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.192+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 0x7fb3fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:02.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.194+0000 7fb41ea00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 0x7fb3fc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cafe0 tx=0x7fb4100074b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:02.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.196+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb418045360 con 0x7fb42015eb00 2024-05-27T12:35:02.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.432+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb4201f62e0 con 0x7fb42015eb00 2024-05-27T12:35:02.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.963+0000 7fb417400640 1 -- 172.21.15.94:0/2172253903 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb41807aba0 con 0x7fb42015eb00 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.966+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 msgr2=0x7fb3fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.966+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 0x7fb3fc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cafe0 tx=0x7fb4100074b0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.966+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 msgr2=0x7fb4201f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.966+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb4180046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 shutdown_connections 2024-05-27T12:35:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3fc06c8f0 0x7fb3fc06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4200d3e50 0x7fb4201601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb42015eb00 0x7fb4201f5da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 --2- 172.21.15.94:0/2172253903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4200d1f40 0x7fb4200b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 >> 172.21.15.94:0/2172253903 conn(0x7fb420067a50 msgr2=0x7fb4200b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 shutdown_connections 2024-05-27T12:35:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:02.967+0000 7fb425e00640 1 -- 172.21.15.94:0/2172253903 wait complete. 2024-05-27T12:35:02.985 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-27T12:35:02.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:03.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.186+0000 7fe908c00640 1 Processor -- start 2024-05-27T12:35:03.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.187+0000 7fe908c00640 1 -- start start 2024-05-27T12:35:03.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.188+0000 7fe908c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe904071f20 0x7fe9040723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.188+0000 7fe908c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 0x7fe904072d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.188+0000 7fe908c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe90415b470 con 0x7fe9041553b0 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.188+0000 7fe908c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe90415b710 con 0x7fe904071f20 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.188+0000 7fe908c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe90415b9f0 con 0x7fe904072910 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.189+0000 7fe902800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 0x7fe904072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.189+0000 7fe901400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe904071f20 0x7fe9040723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.189+0000 7fe901400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe904071f20 0x7fe9040723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:58930/0 (socket says 172.21.15.94:58930) 2024-05-27T12:35:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.189+0000 7fe902800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 0x7fe904072d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57762/0 (socket says 172.21.15.94:57762) 2024-05-27T12:35:03.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.189+0000 7fe901400640 1 -- 172.21.15.94:0/2437601687 learned_addr learned my addr 172.21.15.94:0/2437601687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:03.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.190+0000 7fe901400640 1 -- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 msgr2=0x7fe904072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:03.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.190+0000 7fe901400640 1 --2- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 0x7fe904072d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.190+0000 7fe901400640 1 -- 172.21.15.94:0/2437601687 >> v1:172.21.15.97:6789/0 conn(0x7fe9041553b0 legacy=0x7fe9040cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.190+0000 7fe901400640 1 -- 172.21.15.94:0/2437601687 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90415bcd0 con 0x7fe904071f20 2024-05-27T12:35:03.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.191+0000 7fe901400640 1 --2- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904071f20 0x7fe9040723d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0bb014de7bc5f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe900a00640 1 -- 172.21.15.94:0/2437601687 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec037070 con 0x7fe904071f20 2024-05-27T12:35:03.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe900a00640 1 -- 172.21.15.94:0/2437601687 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec032440 con 0x7fe904071f20 2024-05-27T12:35:03.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe900a00640 1 -- 172.21.15.94:0/2437601687 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec032810 con 0x7fe904071f20 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 -- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904071f20 msgr2=0x7fe9040723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 --2- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904071f20 0x7fe9040723d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec02a1c0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 -- 172.21.15.94:0/2437601687 shutdown_connections 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 --2- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904072910 0x7fe904072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 --2- 172.21.15.94:0/2437601687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe904071f20 0x7fe9040723d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.192+0000 7fe908c00640 1 -- 172.21.15.94:0/2437601687 >> 172.21.15.94:0/2437601687 conn(0x7fe9040b3830 msgr2=0x7fe9040b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.193+0000 7fe908c00640 1 -- 172.21.15.94:0/2437601687 shutdown_connections 2024-05-27T12:35:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.193+0000 7fe908c00640 1 -- 172.21.15.94:0/2437601687 wait complete. 2024-05-27T12:35:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.195+0000 7fe908c00640 1 Processor -- start 2024-05-27T12:35:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.195+0000 7fe908c00640 1 -- start start 2024-05-27T12:35:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904071f20 0x7fe9041461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9041553b0 0x7fe9041539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040c58f0 con 0x7fe904072910 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe904155790 con 0x7fe9041553b0 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe908c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe9040d0430 con 0x7fe904071f20 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe901400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe901400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56342/0 (socket says 172.21.15.94:56342) 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.196+0000 7fe901400640 1 -- 172.21.15.94:0/1717029756 learned_addr learned my addr 172.21.15.94:0/1717029756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901e00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904071f20 0x7fe9041461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 -- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904071f20 msgr2=0x7fe9041461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe902800640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9041553b0 0x7fe9041539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904071f20 0x7fe9041461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 -- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9041553b0 msgr2=0x7fe9041539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9041553b0 0x7fe9041539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 -- 172.21.15.94:0/1717029756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90415c470 con 0x7fe904072910 2024-05-27T12:35:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe901400640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000c00 tx=0x7fe8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec037040 con 0x7fe904072910 2024-05-27T12:35:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec042070 con 0x7fe904072910 2024-05-27T12:35:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.197+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90415c750 con 0x7fe904072910 2024-05-27T12:35:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.198+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec030880 con 0x7fe904072910 2024-05-27T12:35:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.198+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90415cd10 con 0x7fe904072910 2024-05-27T12:35:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.199+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe904075460 con 0x7fe904072910 2024-05-27T12:35:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.200+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8ec02ad10 con 0x7fe904072910 2024-05-27T12:35:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.200+0000 7fe8fa000640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.201+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8ec0abff0 con 0x7fe904072910 2024-05-27T12:35:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.201+0000 7fe901e00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.202+0000 7fe901e00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 0x7fe8e006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000f60 tx=0x7fe8f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.205+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8ec07a470 con 0x7fe904072910 2024-05-27T12:35:03.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.441+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9040cd740 con 0x7fe904072910 2024-05-27T12:35:03.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.442+0000 7fe8fa000640 1 -- 172.21.15.94:0/1717029756 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7fe8ec079d70 con 0x7fe904072910 2024-05-27T12:35:03.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:03.442 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 msgr2=0x7fe8e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 0x7fe8e006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000f60 tx=0x7fe8f4009040 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 msgr2=0x7fe904156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000c00 tx=0x7fe8ec0046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 shutdown_connections 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe8e006c8f0 0x7fe8e006edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9041553b0 0x7fe9041539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe904072910 0x7fe904156630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 --2- 172.21.15.94:0/1717029756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe904071f20 0x7fe9041461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 >> 172.21.15.94:0/1717029756 conn(0x7fe9040b3830 msgr2=0x7fe90414a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.444+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 shutdown_connections 2024-05-27T12:35:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.445+0000 7fe908c00640 1 -- 172.21.15.94:0/1717029756 wait complete. 2024-05-27T12:35:03.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.654+0000 7f7d72000640 1 Processor -- start 2024-05-27T12:35:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.654+0000 7f7d72000640 1 -- start start 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.655+0000 7f7d72000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d6c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.655+0000 7f7d72000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0ceae0 con 0x7f7d6c071d60 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d72000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c1602c0 con 0x7f7d6c145cf0 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d72000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c1605a0 con 0x7f7d6c1460d0 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d71600640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7d6c071d60 0x7f7d6c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48046/0 (socket says 172.21.15.94:48046) 2024-05-27T12:35:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d71600640 1 -- 172.21.15.94:0/2548254987 learned_addr learned my addr 172.21.15.94:0/2548254987 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d63e00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d6c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2294917376 0 0) 0x7f7d6c0ceae0 con 0x7f7d6c071d60 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.656+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d4c003520 con 0x7f7d6c071d60 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2430188415 0 0) 0x7f7d4c003520 con 0x7f7d6c071d60 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 msgr2=0x7f7d6c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d6c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 >> v1:172.21.15.97:6789/0 conn(0x7f7d6c145cf0 legacy=0x7f7d6c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c160880 con 0x7f7d6c071d60 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f7d640019f0 con 0x7f7d6c071d60 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 >> v1:172.21.15.94:6789/0 conn(0x7f7d6c071d60 legacy=0x7f7d6c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4c0039e0 0x7f7d4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d4c004800 0x7f7d4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1605a0 con 0x7f7d4c004800 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c00ba70 con 0x7f7d4c0039e0 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4c00bc70 con 0x7f7d6c1460d0 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d63e00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d4c004800 0x7f7d4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d70c00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4c0039e0 0x7f7d4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d71600640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d63e00640 1 -- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 msgr2=0x7f7d4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d63e00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.657+0000 7f7d63e00640 1 -- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4c0039e0 msgr2=0x7f7d4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:03.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.658+0000 7f7d63e00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4c0039e0 0x7f7d4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.658+0000 7f7d63e00640 1 -- 172.21.15.94:0/2548254987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c003520 con 0x7f7d4c004800 2024-05-27T12:35:03.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.658+0000 7f7d71600640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d6c1460d0 0x7f7d4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.658+0000 7f7d63e00640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d4c004800 0x7f7d4c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009a90 tx=0x7f7d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2198f246fa68676 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c036070 con 0x7f7d4c004800 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c031400 con 0x7f7d4c004800 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d6be00640 1 -- 172.21.15.94:0/2548254987 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c009690 con 0x7f7d4c004800 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 -- 172.21.15.94:0/2548254987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d4c004800 msgr2=0x7f7d4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 --2- 172.21.15.94:0/2548254987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d4c004800 0x7f7d4c004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009a90 tx=0x7f7d4c005830 comp rx=0 tx=0).stop 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d71600640 1 -- 172.21.15.94:0/2548254987 reap_dead start 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 -- 172.21.15.94:0/2548254987 shutdown_connections 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 -- 172.21.15.94:0/2548254987 >> 172.21.15.94:0/2548254987 conn(0x7f7d6c0b2160 msgr2=0x7f7d6c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:03.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 -- 172.21.15.94:0/2548254987 shutdown_connections 2024-05-27T12:35:03.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.659+0000 7f7d72000640 1 -- 172.21.15.94:0/2548254987 wait complete. 2024-05-27T12:35:03.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.661+0000 7f7d72000640 1 Processor -- start 2024-05-27T12:35:03.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.661+0000 7f7d72000640 1 -- start start 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 0x7f7d6c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 0x7f7d6c0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c06eb00 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0cedc0 con 0x7f7d6c0ae130 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.662+0000 7f7d72000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c06ec80 con 0x7f7d4c004800 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56362/0 (socket says 172.21.15.94:56362) 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 -- 172.21.15.94:0/473346722 learned_addr learned my addr 172.21.15.94:0/473346722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d70c00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 0x7f7d6c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d71600640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 0x7f7d6c0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 -- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 msgr2=0x7f7d6c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 0x7f7d6c0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 -- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 msgr2=0x7f7d6c0ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 0x7f7d6c0ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 -- 172.21.15.94:0/473346722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0b0f00 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d70c00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 0x7f7d6c0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d71600640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 0x7f7d6c0ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.663+0000 7f7d63e00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c034040 tx=0x7f7d5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.664+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c040020 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.664+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c003bc0 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.664+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c0b11e0 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.664+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c036040 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.664+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c0a9610 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.665+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c075460 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.666+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d5c043020 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.667+0000 7f7d69600640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 0x7f7d4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:03.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.667+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d5c0abd20 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.667+0000 7f7d70c00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 0x7f7d4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:03.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.668+0000 7f7d70c00640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 0x7f7d4806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d54001630 tx=0x7f7d54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.672+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d5c04a070 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.874+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d6c144d00 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.877+0000 7f7d69600640 1 -- 172.21.15.94:0/473346722 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f7d5c0044e0 con 0x7f7d6c0b03b0 2024-05-27T12:35:03.877 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:03.877 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 msgr2=0x7f7d4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 0x7f7d4806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d54001630 tx=0x7f7d54009040 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 msgr2=0x7f7d6c0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c034040 tx=0x7f7d5c004730 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 shutdown_connections 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4806c920 0x7f7d4806ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d6c0ae130 0x7f7d6c0ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d6c0b03b0 0x7f7d6c0b0830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 --2- 172.21.15.94:0/473346722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d4c004800 0x7f7d6c0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 >> 172.21.15.94:0/473346722 conn(0x7f7d6c0b2160 msgr2=0x7f7d6c0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 shutdown_connections 2024-05-27T12:35:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:03.880+0000 7f7d72000640 1 -- 172.21.15.94:0/473346722 wait complete. 2024-05-27T12:35:03.898 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-27T12:35:03.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:35:04.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.096+0000 7f2601800640 1 Processor -- start 2024-05-27T12:35:04.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.096+0000 7f2601800640 1 -- start start 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2601800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc06f1f0 0x7f25fc0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2601800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f25fc0cfcd0 0x7f25fc0725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2601800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc06f860 con 0x7f25fc15ac60 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2601800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f25fc0cf300 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2601800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc0cecb0 con 0x7f25fc06f1f0 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f25fc0cfcd0 0x7f25fc0725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f25fc0cfcd0 0x7f25fc0725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56378/0 (socket says 172.21.15.94:56378) 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 -- 172.21.15.94:0/3028983474 learned_addr learned my addr 172.21.15.94:0/3028983474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f25fb400640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc06f1f0 0x7f25fc0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 -- 172.21.15.94:0/3028983474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc06f1f0 msgr2=0x7f25fc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc06f1f0 0x7f25fc0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 -- 172.21.15.94:0/3028983474 >> v1:172.21.15.97:6789/0 conn(0x7f25fc15ac60 legacy=0x7f25fc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:04.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.098+0000 7f2600e00640 1 -- 172.21.15.94:0/3028983474 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0cef90 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.099+0000 7f2600e00640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc0725f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009e50 tx=0x7f25f402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af62ba0037d429f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f25faa00640 1 -- 172.21.15.94:0/3028983474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f40333f0 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f25faa00640 1 -- 172.21.15.94:0/3028983474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f4033590 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f25faa00640 1 -- 172.21.15.94:0/3028983474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4033960 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 -- 172.21.15.94:0/3028983474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc0cfcd0 msgr2=0x7f25fc0725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc0725f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009e50 tx=0x7f25f402c8e0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 -- 172.21.15.94:0/3028983474 shutdown_connections 2024-05-27T12:35:04.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc0725f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 --2- 172.21.15.94:0/3028983474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc06f1f0 0x7f25fc0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 -- 172.21.15.94:0/3028983474 >> 172.21.15.94:0/3028983474 conn(0x7f25fc0b2160 msgr2=0x7f25fc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:04.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.100+0000 7f2601800640 1 -- 172.21.15.94:0/3028983474 shutdown_connections 2024-05-27T12:35:04.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.101+0000 7f2601800640 1 -- 172.21.15.94:0/3028983474 wait complete. 2024-05-27T12:35:04.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.102+0000 7f2601800640 1 Processor -- start 2024-05-27T12:35:04.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.102+0000 7f2601800640 1 -- start start 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc15ac60 0x7f25fc152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc072cb0 con 0x7f25fc06f1f0 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc072b30 con 0x7f25fc0cfcd0 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f2601800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc072e30 con 0x7f25fc15ac60 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f25fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56386/0 (socket says 172.21.15.94:56386) 2024-05-27T12:35:04.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.103+0000 7f25fbe00640 1 -- 172.21.15.94:0/2753981829 learned_addr learned my addr 172.21.15.94:0/2753981829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 -- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc15ac60 msgr2=0x7f25fc152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc15ac60 0x7f25fc152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 -- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25fc0cfcd0 msgr2=0x7f25fc14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 -- 172.21.15.94:0/2753981829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc14fe60 con 0x7f25fc06f1f0 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25fbe00640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c460 tx=0x7f25ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec016020 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec012870 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec012c60 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.104+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc15c0f0 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.105+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc15c390 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.106+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc075460 con 0x7f25fc06f1f0 2024-05-27T12:35:04.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.106+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25ec019020 con 0x7f25fc06f1f0 2024-05-27T12:35:04.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.107+0000 7f25f3e00640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 0x7f25d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.107+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f25ec08b640 con 0x7f25fc06f1f0 2024-05-27T12:35:04.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.108+0000 7f25fb400640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 0x7f25d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.109+0000 7f25fb400640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 0x7f25d806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0b01d0 tx=0x7f25e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.112+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25ec020070 con 0x7f25fc06f1f0 2024-05-27T12:35:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.294+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25fc15c810 con 0x7f25d806c920 2024-05-27T12:35:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.317+0000 7f25f3e00640 1 -- 172.21.15.94:0/2753981829 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f25fc15c810 con 0x7f25d806c920 2024-05-27T12:35:04.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:04.322 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:35:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 msgr2=0x7f25d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 0x7f25d806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0b01d0 tx=0x7f25e4008040 comp rx=0 tx=0).stop 2024-05-27T12:35:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 msgr2=0x7f25fc150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c460 tx=0x7f25ec00c920 comp rx=0 tx=0).stop 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 shutdown_connections 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f25d806c920 0x7f25d806ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25fc0cfcd0 0x7f25fc14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25fc15ac60 0x7f25fc152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 --2- 172.21.15.94:0/2753981829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25fc06f1f0 0x7f25fc150970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 >> 172.21.15.94:0/2753981829 conn(0x7f25fc0b2160 msgr2=0x7f25fc0af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 shutdown_connections 2024-05-27T12:35:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.323+0000 7f2601800640 1 -- 172.21.15.94:0/2753981829 wait complete. 2024-05-27T12:35:04.340 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-27T12:35:02.859161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.983202"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982878+0000","last_change":"2024-05-27T12:34:59.958231+0000","last_active":"2024-05-27T12:35:00.982878+0000","last_peered":"2024-05-27T12:35:00.982878+0000","last_clean":"2024-05-27T12:35:00.982878+0000","last_became_active":"2024-05-27T12:34:59.957423+0000","last_became_peered":"2024-05-27T12:34:59.957423+0000","last_unstale":"2024-05-27T12:35:00.982878+0000","last_undegraded":"2024-05-27T12:35:00.982878+0000","last_fullsized":"2024-05-27T12:35:00.982878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944746+0000","last_change":"2024-05-27T12:34:59.945994+0000","last_active":"2024-05-27T12:35:00.944746+0000","last_peered":"2024-05-27T12:35:00.944746+0000","last_clean":"2024-05-27T12:35:00.944746+0000","last_became_active":"2024-05-27T12:34:59.945737+0000","last_became_peered":"2024-05-27T12:34:59.945737+0000","last_unstale":"2024-05-27T12:35:00.944746+0000","last_undegraded":"2024-05-27T12:35:00.944746+0000","last_fullsized":"2024-05-27T12:35:00.944746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983788+0000","last_change":"2024-05-27T12:34:59.941922+0000","last_active":"2024-05-27T12:35:00.983788+0000","last_peered":"2024-05-27T12:35:00.983788+0000","last_clean":"2024-05-27T12:35:00.983788+0000","last_became_active":"2024-05-27T12:34:59.941543+0000","last_became_peered":"2024-05-27T12:34:59.941543+0000","last_unstale":"2024-05-27T12:35:00.983788+0000","last_undegraded":"2024-05-27T12:35:00.983788+0000","last_fullsized":"2024-05-27T12:35:00.983788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979987+0000","last_change":"2024-05-27T12:34:59.946464+0000","last_active":"2024-05-27T12:35:00.979987+0000","last_peered":"2024-05-27T12:35:00.979987+0000","last_clean":"2024-05-27T12:35:00.979987+0000","last_became_active":"2024-05-27T12:34:59.943307+0000","last_became_peered":"2024-05-27T12:34:59.943307+0000","last_unstale":"2024-05-27T12:35:00.979987+0000","last_undegraded":"2024-05-27T12:35:00.979987+0000","last_fullsized":"2024-05-27T12:35:00.979987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982559+0000","last_change":"2024-05-27T12:34:59.953536+0000","last_active":"2024-05-27T12:35:00.982559+0000","last_peered":"2024-05-27T12:35:00.982559+0000","last_clean":"2024-05-27T12:35:00.982559+0000","last_became_active":"2024-05-27T12:34:59.953266+0000","last_became_peered":"2024-05-27T12:34:59.953266+0000","last_unstale":"2024-05-27T12:35:00.982559+0000","last_undegraded":"2024-05-27T12:35:00.982559+0000","last_fullsized":"2024-05-27T12:35:00.982559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979484+0000","last_change":"2024-05-27T12:34:59.939835+0000","last_active":"2024-05-27T12:35:00.979484+0000","last_peered":"2024-05-27T12:35:00.979484+0000","last_clean":"2024-05-27T12:35:00.979484+0000","last_became_active":"2024-05-27T12:34:59.938363+0000","last_became_peered":"2024-05-27T12:34:59.938363+0000","last_unstale":"2024-05-27T12:35:00.979484+0000","last_undegraded":"2024-05-27T12:35:00.979484+0000","last_fullsized":"2024-05-27T12:35:00.979484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981481+0000","last_change":"2024-05-27T12:34:59.949943+0000","last_active":"2024-05-27T12:35:00.981481+0000","last_peered":"2024-05-27T12:35:00.981481+0000","last_clean":"2024-05-27T12:35:00.981481+0000","last_became_active":"2024-05-27T12:34:59.949473+0000","last_became_peered":"2024-05-27T12:34:59.949473+0000","last_unstale":"2024-05-27T12:35:00.981481+0000","last_undegraded":"2024-05-27T12:35:00.981481+0000","last_fullsized":"2024-05-27T12:35:00.981481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980190+0000","last_change":"2024-05-27T12:34:59.954206+0000","last_active":"2024-05-27T12:35:00.980190+0000","last_peered":"2024-05-27T12:35:00.980190+0000","last_clean":"2024-05-27T12:35:00.980190+0000","last_became_active":"2024-05-27T12:34:59.953560+0000","last_became_peered":"2024-05-27T12:34:59.953560+0000","last_unstale":"2024-05-27T12:35:00.980190+0000","last_undegraded":"2024-05-27T12:35:00.980190+0000","last_fullsized":"2024-05-27T12:35:00.980190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982883+0000","last_change":"2024-05-27T12:34:59.960424+0000","last_active":"2024-05-27T12:35:00.982883+0000","last_peered":"2024-05-27T12:35:00.982883+0000","last_clean":"2024-05-27T12:35:00.982883+0000","last_became_active":"2024-05-27T12:34:59.959909+0000","last_became_peered":"2024-05-27T12:34:59.959909+0000","last_unstale":"2024-05-27T12:35:00.982883+0000","last_undegraded":"2024-05-27T12:35:00.982883+0000","last_fullsized":"2024-05-27T12:35:00.982883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945456+0000","last_change":"2024-05-27T12:34:59.953459+0000","last_active":"2024-05-27T12:35:00.945456+0000","last_peered":"2024-05-27T12:35:00.945456+0000","last_clean":"2024-05-27T12:35:00.945456+0000","last_became_active":"2024-05-27T12:34:59.947619+0000","last_became_peered":"2024-05-27T12:34:59.947619+0000","last_unstale":"2024-05-27T12:35:00.945456+0000","last_undegraded":"2024-05-27T12:35:00.945456+0000","last_fullsized":"2024-05-27T12:35:00.945456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945452+0000","last_change":"2024-05-27T12:34:59.956011+0000","last_active":"2024-05-27T12:35:00.945452+0000","last_peered":"2024-05-27T12:35:00.945452+0000","last_clean":"2024-05-27T12:35:00.945452+0000","last_became_active":"2024-05-27T12:34:59.955642+0000","last_became_peered":"2024-05-27T12:34:59.955642+0000","last_unstale":"2024-05-27T12:35:00.945452+0000","last_undegraded":"2024-05-27T12:35:00.945452+0000","last_fullsized":"2024-05-27T12:35:00.945452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947001+0000","last_change":"2024-05-27T12:34:59.948810+0000","last_active":"2024-05-27T12:35:00.947001+0000","last_peered":"2024-05-27T12:35:00.947001+0000","last_clean":"2024-05-27T12:35:00.947001+0000","last_became_active":"2024-05-27T12:34:59.948353+0000","last_became_peered":"2024-05-27T12:34:59.948353+0000","last_unstale":"2024-05-27T12:35:00.947001+0000","last_undegraded":"2024-05-27T12:35:00.947001+0000","last_fullsized":"2024-05-27T12:35:00.947001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:52.789382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981338+0000","last_change":"2024-05-27T12:34:59.944238+0000","last_active":"2024-05-27T12:35:00.981338+0000","last_peered":"2024-05-27T12:35:00.981338+0000","last_clean":"2024-05-27T12:35:00.981338+0000","last_became_active":"2024-05-27T12:34:59.943490+0000","last_became_peered":"2024-05-27T12:34:59.943490+0000","last_unstale":"2024-05-27T12:35:00.981338+0000","last_undegraded":"2024-05-27T12:35:00.981338+0000","last_fullsized":"2024-05-27T12:35:00.981338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983782+0000","last_change":"2024-05-27T12:34:59.956775+0000","last_active":"2024-05-27T12:35:00.983782+0000","last_peered":"2024-05-27T12:35:00.983782+0000","last_clean":"2024-05-27T12:35:00.983782+0000","last_became_active":"2024-05-27T12:34:59.954558+0000","last_became_peered":"2024-05-27T12:34:59.954558+0000","last_unstale":"2024-05-27T12:35:00.983782+0000","last_undegraded":"2024-05-27T12:35:00.983782+0000","last_fullsized":"2024-05-27T12:35:00.983782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944437+0000","last_change":"2024-05-27T12:34:59.952264+0000","last_active":"2024-05-27T12:35:00.944437+0000","last_peered":"2024-05-27T12:35:00.944437+0000","last_clean":"2024-05-27T12:35:00.944437+0000","last_became_active":"2024-05-27T12:34:59.951960+0000","last_became_peered":"2024-05-27T12:34:59.951960+0000","last_unstale":"2024-05-27T12:35:00.944437+0000","last_undegraded":"2024-05-27T12:35:00.944437+0000","last_fullsized":"2024-05-27T12:35:00.944437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979305+0000","last_change":"2024-05-27T12:34:59.936327+0000","last_active":"2024-05-27T12:35:00.979305+0000","last_peered":"2024-05-27T12:35:00.979305+0000","last_clean":"2024-05-27T12:35:00.979305+0000","last_became_active":"2024-05-27T12:34:59.935576+0000","last_became_peered":"2024-05-27T12:34:59.935576+0000","last_unstale":"2024-05-27T12:35:00.979305+0000","last_undegraded":"2024-05-27T12:35:00.979305+0000","last_fullsized":"2024-05-27T12:35:00.979305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980066+0000","last_change":"2024-05-27T12:34:59.956063+0000","last_active":"2024-05-27T12:35:00.980066+0000","last_peered":"2024-05-27T12:35:00.980066+0000","last_clean":"2024-05-27T12:35:00.980066+0000","last_became_active":"2024-05-27T12:34:59.955763+0000","last_became_peered":"2024-05-27T12:34:59.955763+0000","last_unstale":"2024-05-27T12:35:00.980066+0000","last_undegraded":"2024-05-27T12:35:00.980066+0000","last_fullsized":"2024-05-27T12:35:00.980066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981742+0000","last_change":"2024-05-27T12:34:59.937118+0000","last_active":"2024-05-27T12:35:00.981742+0000","last_peered":"2024-05-27T12:35:00.981742+0000","last_clean":"2024-05-27T12:35:00.981742+0000","last_became_active":"2024-05-27T12:34:59.936814+0000","last_became_peered":"2024-05-27T12:34:59.936814+0000","last_unstale":"2024-05-27T12:35:00.981742+0000","last_undegraded":"2024-05-27T12:35:00.981742+0000","last_fullsized":"2024-05-27T12:35:00.981742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:54.133638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983099+0000","last_change":"2024-05-27T12:34:59.952246+0000","last_active":"2024-05-27T12:35:00.983099+0000","last_peered":"2024-05-27T12:35:00.983099+0000","last_clean":"2024-05-27T12:35:00.983099+0000","last_became_active":"2024-05-27T12:34:59.951939+0000","last_became_peered":"2024-05-27T12:34:59.951939+0000","last_unstale":"2024-05-27T12:35:00.983099+0000","last_undegraded":"2024-05-27T12:35:00.983099+0000","last_fullsized":"2024-05-27T12:35:00.983099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945011+0000","last_change":"2024-05-27T12:34:59.947085+0000","last_active":"2024-05-27T12:35:00.945011+0000","last_peered":"2024-05-27T12:35:00.945011+0000","last_clean":"2024-05-27T12:35:00.945011+0000","last_became_active":"2024-05-27T12:34:59.946740+0000","last_became_peered":"2024-05-27T12:34:59.946740+0000","last_unstale":"2024-05-27T12:35:00.945011+0000","last_undegraded":"2024-05-27T12:35:00.945011+0000","last_fullsized":"2024-05-27T12:35:00.945011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982971+0000","last_change":"2024-05-27T12:34:59.956268+0000","last_active":"2024-05-27T12:35:00.982971+0000","last_peered":"2024-05-27T12:35:00.982971+0000","last_clean":"2024-05-27T12:35:00.982971+0000","last_became_active":"2024-05-27T12:34:59.950964+0000","last_became_peered":"2024-05-27T12:34:59.950964+0000","last_unstale":"2024-05-27T12:35:00.982971+0000","last_undegraded":"2024-05-27T12:35:00.982971+0000","last_fullsized":"2024-05-27T12:35:00.982971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:12.683859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979683+0000","last_change":"2024-05-27T12:34:59.956010+0000","last_active":"2024-05-27T12:35:00.979683+0000","last_peered":"2024-05-27T12:35:00.979683+0000","last_clean":"2024-05-27T12:35:00.979683+0000","last_became_active":"2024-05-27T12:34:59.955692+0000","last_became_peered":"2024-05-27T12:34:59.955692+0000","last_unstale":"2024-05-27T12:35:00.979683+0000","last_undegraded":"2024-05-27T12:35:00.979683+0000","last_fullsized":"2024-05-27T12:35:00.979683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980587+0000","last_change":"2024-05-27T12:34:59.950671+0000","last_active":"2024-05-27T12:35:00.980587+0000","last_peered":"2024-05-27T12:35:00.980587+0000","last_clean":"2024-05-27T12:35:00.980587+0000","last_became_active":"2024-05-27T12:34:59.950168+0000","last_became_peered":"2024-05-27T12:34:59.950168+0000","last_unstale":"2024-05-27T12:35:00.980587+0000","last_undegraded":"2024-05-27T12:35:00.980587+0000","last_fullsized":"2024-05-27T12:35:00.980587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979458+0000","last_change":"2024-05-27T12:34:59.947016+0000","last_active":"2024-05-27T12:35:00.979458+0000","last_peered":"2024-05-27T12:35:00.979458+0000","last_clean":"2024-05-27T12:35:00.979458+0000","last_became_active":"2024-05-27T12:34:59.946710+0000","last_became_peered":"2024-05-27T12:34:59.946710+0000","last_unstale":"2024-05-27T12:35:00.979458+0000","last_undegraded":"2024-05-27T12:35:00.979458+0000","last_fullsized":"2024-05-27T12:35:00.979458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982935+0000","last_change":"2024-05-27T12:34:59.952637+0000","last_active":"2024-05-27T12:35:00.982935+0000","last_peered":"2024-05-27T12:35:00.982935+0000","last_clean":"2024-05-27T12:35:00.982935+0000","last_became_active":"2024-05-27T12:34:59.951346+0000","last_became_peered":"2024-05-27T12:34:59.951346+0000","last_unstale":"2024-05-27T12:35:00.982935+0000","last_undegraded":"2024-05-27T12:35:00.982935+0000","last_fullsized":"2024-05-27T12:35:00.982935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945571+0000","last_change":"2024-05-27T12:34:59.958301+0000","last_active":"2024-05-27T12:35:00.945571+0000","last_peered":"2024-05-27T12:35:00.945571+0000","last_clean":"2024-05-27T12:35:00.945571+0000","last_became_active":"2024-05-27T12:34:59.957996+0000","last_became_peered":"2024-05-27T12:34:59.957996+0000","last_unstale":"2024-05-27T12:35:00.945571+0000","last_undegraded":"2024-05-27T12:35:00.945571+0000","last_fullsized":"2024-05-27T12:35:00.945571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944292+0000","last_change":"2024-05-27T12:34:59.949515+0000","last_active":"2024-05-27T12:35:00.944292+0000","last_peered":"2024-05-27T12:35:00.944292+0000","last_clean":"2024-05-27T12:35:00.944292+0000","last_became_active":"2024-05-27T12:34:59.945960+0000","last_became_peered":"2024-05-27T12:34:59.945960+0000","last_unstale":"2024-05-27T12:35:00.944292+0000","last_undegraded":"2024-05-27T12:35:00.944292+0000","last_fullsized":"2024-05-27T12:35:00.944292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980945+0000","last_change":"2024-05-27T12:34:59.950471+0000","last_active":"2024-05-27T12:35:00.980945+0000","last_peered":"2024-05-27T12:35:00.980945+0000","last_clean":"2024-05-27T12:35:00.980945+0000","last_became_active":"2024-05-27T12:34:59.950135+0000","last_became_peered":"2024-05-27T12:34:59.950135+0000","last_unstale":"2024-05-27T12:35:00.980945+0000","last_undegraded":"2024-05-27T12:35:00.980945+0000","last_fullsized":"2024-05-27T12:35:00.980945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983332+0000","last_change":"2024-05-27T12:34:59.942121+0000","last_active":"2024-05-27T12:35:00.983332+0000","last_peered":"2024-05-27T12:35:00.983332+0000","last_clean":"2024-05-27T12:35:00.983332+0000","last_became_active":"2024-05-27T12:34:59.941006+0000","last_became_peered":"2024-05-27T12:34:59.941006+0000","last_unstale":"2024-05-27T12:35:00.983332+0000","last_undegraded":"2024-05-27T12:35:00.983332+0000","last_fullsized":"2024-05-27T12:35:00.983332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979945+0000","last_change":"2024-05-27T12:34:59.957178+0000","last_active":"2024-05-27T12:35:00.979945+0000","last_peered":"2024-05-27T12:35:00.979945+0000","last_clean":"2024-05-27T12:35:00.979945+0000","last_became_active":"2024-05-27T12:34:59.956854+0000","last_became_peered":"2024-05-27T12:34:59.956854+0000","last_unstale":"2024-05-27T12:35:00.979945+0000","last_undegraded":"2024-05-27T12:35:00.979945+0000","last_fullsized":"2024-05-27T12:35:00.979945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980174+0000","last_change":"2024-05-27T12:34:59.954564+0000","last_active":"2024-05-27T12:35:00.980174+0000","last_peered":"2024-05-27T12:35:00.980174+0000","last_clean":"2024-05-27T12:35:00.980174+0000","last_became_active":"2024-05-27T12:34:59.953442+0000","last_became_peered":"2024-05-27T12:34:59.953442+0000","last_unstale":"2024-05-27T12:35:00.980174+0000","last_undegraded":"2024-05-27T12:35:00.980174+0000","last_fullsized":"2024-05-27T12:35:00.980174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946038+0000","last_change":"2024-05-27T12:34:59.936968+0000","last_active":"2024-05-27T12:35:00.946038+0000","last_peered":"2024-05-27T12:35:00.946038+0000","last_clean":"2024-05-27T12:35:00.946038+0000","last_became_active":"2024-05-27T12:34:59.935850+0000","last_became_peered":"2024-05-27T12:34:59.935850+0000","last_unstale":"2024-05-27T12:35:00.946038+0000","last_undegraded":"2024-05-27T12:35:00.946038+0000","last_fullsized":"2024-05-27T12:35:00.946038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947845+0000","last_change":"2024-05-27T12:34:59.955463+0000","last_active":"2024-05-27T12:35:00.947845+0000","last_peered":"2024-05-27T12:35:00.947845+0000","last_clean":"2024-05-27T12:35:00.947845+0000","last_became_active":"2024-05-27T12:34:59.952053+0000","last_became_peered":"2024-05-27T12:34:59.952053+0000","last_unstale":"2024-05-27T12:35:00.947845+0000","last_undegraded":"2024-05-27T12:35:00.947845+0000","last_fullsized":"2024-05-27T12:35:00.947845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981586+0000","last_change":"2024-05-27T12:34:58.926344+0000","last_active":"2024-05-27T12:35:00.981586+0000","last_peered":"2024-05-27T12:35:00.981586+0000","last_clean":"2024-05-27T12:35:00.981586+0000","last_became_active":"2024-05-27T12:34:58.926021+0000","last_became_peered":"2024-05-27T12:34:58.926021+0000","last_unstale":"2024-05-27T12:35:00.981586+0000","last_undegraded":"2024-05-27T12:35:00.981586+0000","last_fullsized":"2024-05-27T12:35:00.981586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:35:59.130525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983155+0000","last_change":"2024-05-27T12:34:59.948432+0000","last_active":"2024-05-27T12:35:00.983155+0000","last_peered":"2024-05-27T12:35:00.983155+0000","last_clean":"2024-05-27T12:35:00.983155+0000","last_became_active":"2024-05-27T12:34:59.946912+0000","last_became_peered":"2024-05-27T12:34:59.946912+0000","last_unstale":"2024-05-27T12:35:00.983155+0000","last_undegraded":"2024-05-27T12:35:00.983155+0000","last_fullsized":"2024-05-27T12:35:00.983155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945458+0000","last_change":"2024-05-27T12:34:58.918241+0000","last_active":"2024-05-27T12:35:00.945458+0000","last_peered":"2024-05-27T12:35:00.945458+0000","last_clean":"2024-05-27T12:35:00.945458+0000","last_became_active":"2024-05-27T12:34:58.917640+0000","last_became_peered":"2024-05-27T12:34:58.917640+0000","last_unstale":"2024-05-27T12:35:00.945458+0000","last_undegraded":"2024-05-27T12:35:00.945458+0000","last_fullsized":"2024-05-27T12:35:00.945458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980740+0000","last_change":"2024-05-27T12:34:58.912568+0000","last_active":"2024-05-27T12:35:00.980740+0000","last_peered":"2024-05-27T12:35:00.980740+0000","last_clean":"2024-05-27T12:35:00.980740+0000","last_became_active":"2024-05-27T12:34:58.912280+0000","last_became_peered":"2024-05-27T12:34:58.912280+0000","last_unstale":"2024-05-27T12:35:00.980740+0000","last_undegraded":"2024-05-27T12:35:00.980740+0000","last_fullsized":"2024-05-27T12:35:00.980740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980543+0000","last_change":"2024-05-27T12:34:59.947954+0000","last_active":"2024-05-27T12:35:00.980543+0000","last_peered":"2024-05-27T12:35:00.980543+0000","last_clean":"2024-05-27T12:35:00.980543+0000","last_became_active":"2024-05-27T12:34:59.947702+0000","last_became_peered":"2024-05-27T12:34:59.947702+0000","last_unstale":"2024-05-27T12:35:00.980543+0000","last_undegraded":"2024-05-27T12:35:00.980543+0000","last_fullsized":"2024-05-27T12:35:00.980543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979570+0000","last_change":"2024-05-27T12:34:58.918329+0000","last_active":"2024-05-27T12:35:00.979570+0000","last_peered":"2024-05-27T12:35:00.979570+0000","last_clean":"2024-05-27T12:35:00.979570+0000","last_became_active":"2024-05-27T12:34:58.918007+0000","last_became_peered":"2024-05-27T12:34:58.918007+0000","last_unstale":"2024-05-27T12:35:00.979570+0000","last_undegraded":"2024-05-27T12:35:00.979570+0000","last_fullsized":"2024-05-27T12:35:00.979570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943753+0000","last_change":"2024-05-27T12:34:59.952134+0000","last_active":"2024-05-27T12:35:00.943753+0000","last_peered":"2024-05-27T12:35:00.943753+0000","last_clean":"2024-05-27T12:35:00.943753+0000","last_became_active":"2024-05-27T12:34:59.951823+0000","last_became_peered":"2024-05-27T12:34:59.951823+0000","last_unstale":"2024-05-27T12:35:00.943753+0000","last_undegraded":"2024-05-27T12:35:00.943753+0000","last_fullsized":"2024-05-27T12:35:00.943753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945424+0000","last_change":"2024-05-27T12:34:58.925614+0000","last_active":"2024-05-27T12:35:00.945424+0000","last_peered":"2024-05-27T12:35:00.945424+0000","last_clean":"2024-05-27T12:35:00.945424+0000","last_became_active":"2024-05-27T12:34:58.925312+0000","last_became_peered":"2024-05-27T12:34:58.925312+0000","last_unstale":"2024-05-27T12:35:00.945424+0000","last_undegraded":"2024-05-27T12:35:00.945424+0000","last_fullsized":"2024-05-27T12:35:00.945424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979670+0000","last_change":"2024-05-27T12:34:59.958727+0000","last_active":"2024-05-27T12:35:00.979670+0000","last_peered":"2024-05-27T12:35:00.979670+0000","last_clean":"2024-05-27T12:35:00.979670+0000","last_became_active":"2024-05-27T12:34:59.958405+0000","last_became_peered":"2024-05-27T12:34:59.958405+0000","last_unstale":"2024-05-27T12:35:00.979670+0000","last_undegraded":"2024-05-27T12:35:00.979670+0000","last_fullsized":"2024-05-27T12:35:00.979670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979703+0000","last_change":"2024-05-27T12:34:58.914733+0000","last_active":"2024-05-27T12:35:00.979703+0000","last_peered":"2024-05-27T12:35:00.979703+0000","last_clean":"2024-05-27T12:35:00.979703+0000","last_became_active":"2024-05-27T12:34:58.914357+0000","last_became_peered":"2024-05-27T12:34:58.914357+0000","last_unstale":"2024-05-27T12:35:00.979703+0000","last_undegraded":"2024-05-27T12:35:00.979703+0000","last_fullsized":"2024-05-27T12:35:00.979703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984178+0000","last_change":"2024-05-27T12:34:59.947773+0000","last_active":"2024-05-27T12:35:00.984178+0000","last_peered":"2024-05-27T12:35:00.984178+0000","last_clean":"2024-05-27T12:35:00.984178+0000","last_became_active":"2024-05-27T12:34:59.946577+0000","last_became_peered":"2024-05-27T12:34:59.946577+0000","last_unstale":"2024-05-27T12:35:00.984178+0000","last_undegraded":"2024-05-27T12:35:00.984178+0000","last_fullsized":"2024-05-27T12:35:00.984178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944069+0000","last_change":"2024-05-27T12:34:58.926880+0000","last_active":"2024-05-27T12:35:00.944069+0000","last_peered":"2024-05-27T12:35:00.944069+0000","last_clean":"2024-05-27T12:35:00.944069+0000","last_became_active":"2024-05-27T12:34:58.926621+0000","last_became_peered":"2024-05-27T12:34:58.926621+0000","last_unstale":"2024-05-27T12:35:00.944069+0000","last_undegraded":"2024-05-27T12:35:00.944069+0000","last_fullsized":"2024-05-27T12:35:00.944069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947431+0000","last_change":"2024-05-27T12:34:59.949465+0000","last_active":"2024-05-27T12:35:00.947431+0000","last_peered":"2024-05-27T12:35:00.947431+0000","last_clean":"2024-05-27T12:35:00.947431+0000","last_became_active":"2024-05-27T12:34:59.949190+0000","last_became_peered":"2024-05-27T12:34:59.949190+0000","last_unstale":"2024-05-27T12:35:00.947431+0000","last_undegraded":"2024-05-27T12:35:00.947431+0000","last_fullsized":"2024-05-27T12:35:00.947431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:27.503111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945133+0000","last_change":"2024-05-27T12:34:58.905974+0000","last_active":"2024-05-27T12:35:00.945133+0000","last_peered":"2024-05-27T12:35:00.945133+0000","last_clean":"2024-05-27T12:35:00.945133+0000","last_became_active":"2024-05-27T12:34:58.905596+0000","last_became_peered":"2024-05-27T12:34:58.905596+0000","last_unstale":"2024-05-27T12:35:00.945133+0000","last_undegraded":"2024-05-27T12:35:00.945133+0000","last_fullsized":"2024-05-27T12:35:00.945133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982755+0000","last_change":"2024-05-27T12:34:59.953412+0000","last_active":"2024-05-27T12:35:00.982755+0000","last_peered":"2024-05-27T12:35:00.982755+0000","last_clean":"2024-05-27T12:35:00.982755+0000","last_became_active":"2024-05-27T12:34:59.953103+0000","last_became_peered":"2024-05-27T12:34:59.953103+0000","last_unstale":"2024-05-27T12:35:00.982755+0000","last_undegraded":"2024-05-27T12:35:00.982755+0000","last_fullsized":"2024-05-27T12:35:00.982755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983281+0000","last_change":"2024-05-27T12:34:58.909953+0000","last_active":"2024-05-27T12:35:00.983281+0000","last_peered":"2024-05-27T12:35:00.983281+0000","last_clean":"2024-05-27T12:35:00.983281+0000","last_became_active":"2024-05-27T12:34:58.909633+0000","last_became_peered":"2024-05-27T12:34:58.909633+0000","last_unstale":"2024-05-27T12:35:00.983281+0000","last_undegraded":"2024-05-27T12:35:00.983281+0000","last_fullsized":"2024-05-27T12:35:00.983281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:39.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983591+0000","last_change":"2024-05-27T12:34:59.955304+0000","last_active":"2024-05-27T12:35:00.983591+0000","last_peered":"2024-05-27T12:35:00.983591+0000","last_clean":"2024-05-27T12:35:00.983591+0000","last_became_active":"2024-05-27T12:34:59.954940+0000","last_became_peered":"2024-05-27T12:34:59.954940+0000","last_unstale":"2024-05-27T12:35:00.983591+0000","last_undegraded":"2024-05-27T12:35:00.983591+0000","last_fullsized":"2024-05-27T12:35:00.983591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980018+0000","last_change":"2024-05-27T12:34:58.914660+0000","last_active":"2024-05-27T12:35:00.980018+0000","last_peered":"2024-05-27T12:35:00.980018+0000","last_clean":"2024-05-27T12:35:00.980018+0000","last_became_active":"2024-05-27T12:34:58.914368+0000","last_became_peered":"2024-05-27T12:34:58.914368+0000","last_unstale":"2024-05-27T12:35:00.980018+0000","last_undegraded":"2024-05-27T12:35:00.980018+0000","last_fullsized":"2024-05-27T12:35:00.980018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982688+0000","last_change":"2024-05-27T12:34:59.951438+0000","last_active":"2024-05-27T12:35:00.982688+0000","last_peered":"2024-05-27T12:35:00.982688+0000","last_clean":"2024-05-27T12:35:00.982688+0000","last_became_active":"2024-05-27T12:34:59.951140+0000","last_became_peered":"2024-05-27T12:34:59.951140+0000","last_unstale":"2024-05-27T12:35:00.982688+0000","last_undegraded":"2024-05-27T12:35:00.982688+0000","last_fullsized":"2024-05-27T12:35:00.982688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944792+0000","last_change":"2024-05-27T12:34:58.918066+0000","last_active":"2024-05-27T12:35:00.944792+0000","last_peered":"2024-05-27T12:35:00.944792+0000","last_clean":"2024-05-27T12:35:00.944792+0000","last_became_active":"2024-05-27T12:34:58.917613+0000","last_became_peered":"2024-05-27T12:34:58.917613+0000","last_unstale":"2024-05-27T12:35:00.944792+0000","last_undegraded":"2024-05-27T12:35:00.944792+0000","last_fullsized":"2024-05-27T12:35:00.944792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944830+0000","last_change":"2024-05-27T12:34:59.943832+0000","last_active":"2024-05-27T12:35:00.944830+0000","last_peered":"2024-05-27T12:35:00.944830+0000","last_clean":"2024-05-27T12:35:00.944830+0000","last_became_active":"2024-05-27T12:34:59.943535+0000","last_became_peered":"2024-05-27T12:34:59.943535+0000","last_unstale":"2024-05-27T12:35:00.944830+0000","last_undegraded":"2024-05-27T12:35:00.944830+0000","last_fullsized":"2024-05-27T12:35:00.944830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979766+0000","last_change":"2024-05-27T12:34:58.897545+0000","last_active":"2024-05-27T12:35:00.979766+0000","last_peered":"2024-05-27T12:35:00.979766+0000","last_clean":"2024-05-27T12:35:00.979766+0000","last_became_active":"2024-05-27T12:34:58.895961+0000","last_became_peered":"2024-05-27T12:34:58.895961+0000","last_unstale":"2024-05-27T12:35:00.979766+0000","last_undegraded":"2024-05-27T12:35:00.979766+0000","last_fullsized":"2024-05-27T12:35:00.979766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981686+0000","last_change":"2024-05-27T12:34:59.947365+0000","last_active":"2024-05-27T12:35:00.981686+0000","last_peered":"2024-05-27T12:35:00.981686+0000","last_clean":"2024-05-27T12:35:00.981686+0000","last_became_active":"2024-05-27T12:34:59.947060+0000","last_became_peered":"2024-05-27T12:34:59.947060+0000","last_unstale":"2024-05-27T12:35:00.981686+0000","last_undegraded":"2024-05-27T12:35:00.981686+0000","last_fullsized":"2024-05-27T12:35:00.981686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946688+0000","last_change":"2024-05-27T12:34:58.910200+0000","last_active":"2024-05-27T12:35:00.946688+0000","last_peered":"2024-05-27T12:35:00.946688+0000","last_clean":"2024-05-27T12:35:00.946688+0000","last_became_active":"2024-05-27T12:34:58.909819+0000","last_became_peered":"2024-05-27T12:34:58.909819+0000","last_unstale":"2024-05-27T12:35:00.946688+0000","last_undegraded":"2024-05-27T12:35:00.946688+0000","last_fullsized":"2024-05-27T12:35:00.946688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983250+0000","last_change":"2024-05-27T12:34:59.960324+0000","last_active":"2024-05-27T12:35:00.983250+0000","last_peered":"2024-05-27T12:35:00.983250+0000","last_clean":"2024-05-27T12:35:00.983250+0000","last_became_active":"2024-05-27T12:34:59.958530+0000","last_became_peered":"2024-05-27T12:34:59.958530+0000","last_unstale":"2024-05-27T12:35:00.983250+0000","last_undegraded":"2024-05-27T12:35:00.983250+0000","last_fullsized":"2024-05-27T12:35:00.983250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:10:04.239389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982017+0000","last_change":"2024-05-27T12:34:58.905547+0000","last_active":"2024-05-27T12:35:00.982017+0000","last_peered":"2024-05-27T12:35:00.982017+0000","last_clean":"2024-05-27T12:35:00.982017+0000","last_became_active":"2024-05-27T12:34:58.904281+0000","last_became_peered":"2024-05-27T12:34:58.904281+0000","last_unstale":"2024-05-27T12:35:00.982017+0000","last_undegraded":"2024-05-27T12:35:00.982017+0000","last_fullsized":"2024-05-27T12:35:00.982017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983204+0000","last_change":"2024-05-27T12:34:59.960325+0000","last_active":"2024-05-27T12:35:00.983204+0000","last_peered":"2024-05-27T12:35:00.983204+0000","last_clean":"2024-05-27T12:35:00.983204+0000","last_became_active":"2024-05-27T12:34:59.959764+0000","last_became_peered":"2024-05-27T12:34:59.959764+0000","last_unstale":"2024-05-27T12:35:00.983204+0000","last_undegraded":"2024-05-27T12:35:00.983204+0000","last_fullsized":"2024-05-27T12:35:00.983204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:26:24.248099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944296+0000","last_change":"2024-05-27T12:34:58.916163+0000","last_active":"2024-05-27T12:35:00.944296+0000","last_peered":"2024-05-27T12:35:00.944296+0000","last_clean":"2024-05-27T12:35:00.944296+0000","last_became_active":"2024-05-27T12:34:58.915823+0000","last_became_peered":"2024-05-27T12:34:58.915823+0000","last_unstale":"2024-05-27T12:35:00.944296+0000","last_undegraded":"2024-05-27T12:35:00.944296+0000","last_fullsized":"2024-05-27T12:35:00.944296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980392+0000","last_change":"2024-05-27T12:34:59.944578+0000","last_active":"2024-05-27T12:35:00.980392+0000","last_peered":"2024-05-27T12:35:00.980392+0000","last_clean":"2024-05-27T12:35:00.980392+0000","last_became_active":"2024-05-27T12:34:59.941815+0000","last_became_peered":"2024-05-27T12:34:59.941815+0000","last_unstale":"2024-05-27T12:35:00.980392+0000","last_undegraded":"2024-05-27T12:35:00.980392+0000","last_fullsized":"2024-05-27T12:35:00.980392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980249+0000","last_change":"2024-05-27T12:34:58.921269+0000","last_active":"2024-05-27T12:35:00.980249+0000","last_peered":"2024-05-27T12:35:00.980249+0000","last_clean":"2024-05-27T12:35:00.980249+0000","last_became_active":"2024-05-27T12:34:58.920929+0000","last_became_peered":"2024-05-27T12:34:58.920929+0000","last_unstale":"2024-05-27T12:35:00.980249+0000","last_undegraded":"2024-05-27T12:35:00.980249+0000","last_fullsized":"2024-05-27T12:35:00.980249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982666+0000","last_change":"2024-05-27T12:34:59.959353+0000","last_active":"2024-05-27T12:35:00.982666+0000","last_peered":"2024-05-27T12:35:00.982666+0000","last_clean":"2024-05-27T12:35:00.982666+0000","last_became_active":"2024-05-27T12:34:59.959097+0000","last_became_peered":"2024-05-27T12:34:59.959097+0000","last_unstale":"2024-05-27T12:35:00.982666+0000","last_undegraded":"2024-05-27T12:35:00.982666+0000","last_fullsized":"2024-05-27T12:35:00.982666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981367+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:35:00.981367+0000","last_peered":"2024-05-27T12:35:00.981367+0000","last_clean":"2024-05-27T12:35:00.981367+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:35:00.981367+0000","last_undegraded":"2024-05-27T12:35:00.981367+0000","last_fullsized":"2024-05-27T12:35:00.981367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979291+0000","last_change":"2024-05-27T12:34:58.920919+0000","last_active":"2024-05-27T12:35:00.979291+0000","last_peered":"2024-05-27T12:35:00.979291+0000","last_clean":"2024-05-27T12:35:00.979291+0000","last_became_active":"2024-05-27T12:34:58.918391+0000","last_became_peered":"2024-05-27T12:34:58.918391+0000","last_unstale":"2024-05-27T12:35:00.979291+0000","last_undegraded":"2024-05-27T12:35:00.979291+0000","last_fullsized":"2024-05-27T12:35:00.979291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981108+0000","last_change":"2024-05-27T12:34:59.959595+0000","last_active":"2024-05-27T12:35:00.981108+0000","last_peered":"2024-05-27T12:35:00.981108+0000","last_clean":"2024-05-27T12:35:00.981108+0000","last_became_active":"2024-05-27T12:34:59.959298+0000","last_became_peered":"2024-05-27T12:34:59.959298+0000","last_unstale":"2024-05-27T12:35:00.981108+0000","last_undegraded":"2024-05-27T12:35:00.981108+0000","last_fullsized":"2024-05-27T12:35:00.981108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981337+0000","last_change":"2024-05-27T12:34:58.926438+0000","last_active":"2024-05-27T12:35:00.981337+0000","last_peered":"2024-05-27T12:35:00.981337+0000","last_clean":"2024-05-27T12:35:00.981337+0000","last_became_active":"2024-05-27T12:34:58.926205+0000","last_became_peered":"2024-05-27T12:34:58.926205+0000","last_unstale":"2024-05-27T12:35:00.981337+0000","last_undegraded":"2024-05-27T12:35:00.981337+0000","last_fullsized":"2024-05-27T12:35:00.981337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982896+0000","last_change":"2024-05-27T12:34:59.947056+0000","last_active":"2024-05-27T12:35:00.982896+0000","last_peered":"2024-05-27T12:35:00.982896+0000","last_clean":"2024-05-27T12:35:00.982896+0000","last_became_active":"2024-05-27T12:34:59.946724+0000","last_became_peered":"2024-05-27T12:34:59.946724+0000","last_unstale":"2024-05-27T12:35:00.982896+0000","last_undegraded":"2024-05-27T12:35:00.982896+0000","last_fullsized":"2024-05-27T12:35:00.982896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:04.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980419+0000","last_change":"2024-05-27T12:34:58.906765+0000","last_active":"2024-05-27T12:35:00.980419+0000","last_peered":"2024-05-27T12:35:00.980419+0000","last_clean":"2024-05-27T12:35:00.980419+0000","last_became_active":"2024-05-27T12:34:58.904674+0000","last_became_peered":"2024-05-27T12:34:58.904674+0000","last_unstale":"2024-05-27T12:35:00.980419+0000","last_undegraded":"2024-05-27T12:35:00.980419+0000","last_fullsized":"2024-05-27T12:35:00.980419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943526+0000","last_change":"2024-05-27T12:34:59.943677+0000","last_active":"2024-05-27T12:35:00.943526+0000","last_peered":"2024-05-27T12:35:00.943526+0000","last_clean":"2024-05-27T12:35:00.943526+0000","last_became_active":"2024-05-27T12:34:59.943293+0000","last_became_peered":"2024-05-27T12:34:59.943293+0000","last_unstale":"2024-05-27T12:35:00.943526+0000","last_undegraded":"2024-05-27T12:35:00.943526+0000","last_fullsized":"2024-05-27T12:35:00.943526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980228+0000","last_change":"2024-05-27T12:34:58.916105+0000","last_active":"2024-05-27T12:35:00.980228+0000","last_peered":"2024-05-27T12:35:00.980228+0000","last_clean":"2024-05-27T12:35:00.980228+0000","last_became_active":"2024-05-27T12:34:58.913880+0000","last_became_peered":"2024-05-27T12:34:58.913880+0000","last_unstale":"2024-05-27T12:35:00.980228+0000","last_undegraded":"2024-05-27T12:35:00.980228+0000","last_fullsized":"2024-05-27T12:35:00.980228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946999+0000","last_change":"2024-05-27T12:34:59.946006+0000","last_active":"2024-05-27T12:35:00.946999+0000","last_peered":"2024-05-27T12:35:00.946999+0000","last_clean":"2024-05-27T12:35:00.946999+0000","last_became_active":"2024-05-27T12:34:59.945730+0000","last_became_peered":"2024-05-27T12:34:59.945730+0000","last_unstale":"2024-05-27T12:35:00.946999+0000","last_undegraded":"2024-05-27T12:35:00.946999+0000","last_fullsized":"2024-05-27T12:35:00.946999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:24.764029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943841+0000","last_change":"2024-05-27T12:34:58.906953+0000","last_active":"2024-05-27T12:35:00.943841+0000","last_peered":"2024-05-27T12:35:00.943841+0000","last_clean":"2024-05-27T12:35:00.943841+0000","last_became_active":"2024-05-27T12:34:58.906672+0000","last_became_peered":"2024-05-27T12:34:58.906672+0000","last_unstale":"2024-05-27T12:35:00.943841+0000","last_undegraded":"2024-05-27T12:35:00.943841+0000","last_fullsized":"2024-05-27T12:35:00.943841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980531+0000","last_change":"2024-05-27T12:34:59.947648+0000","last_active":"2024-05-27T12:35:00.980531+0000","last_peered":"2024-05-27T12:35:00.980531+0000","last_clean":"2024-05-27T12:35:00.980531+0000","last_became_active":"2024-05-27T12:34:59.946323+0000","last_became_peered":"2024-05-27T12:34:59.946323+0000","last_unstale":"2024-05-27T12:35:00.980531+0000","last_undegraded":"2024-05-27T12:35:00.980531+0000","last_fullsized":"2024-05-27T12:35:00.980531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983010+0000","last_change":"2024-05-27T12:34:58.922207+0000","last_active":"2024-05-27T12:35:00.983010+0000","last_peered":"2024-05-27T12:35:00.983010+0000","last_clean":"2024-05-27T12:35:00.983010+0000","last_became_active":"2024-05-27T12:34:58.921862+0000","last_became_peered":"2024-05-27T12:34:58.921862+0000","last_unstale":"2024-05-27T12:35:00.983010+0000","last_undegraded":"2024-05-27T12:35:00.983010+0000","last_fullsized":"2024-05-27T12:35:00.983010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:44.514186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943666+0000","last_change":"2024-05-27T12:34:59.945417+0000","last_active":"2024-05-27T12:35:00.943666+0000","last_peered":"2024-05-27T12:35:00.943666+0000","last_clean":"2024-05-27T12:35:00.943666+0000","last_became_active":"2024-05-27T12:34:59.945068+0000","last_became_peered":"2024-05-27T12:34:59.945068+0000","last_unstale":"2024-05-27T12:35:00.943666+0000","last_undegraded":"2024-05-27T12:35:00.943666+0000","last_fullsized":"2024-05-27T12:35:00.943666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.965456+0000","last_change":"2024-05-27T12:34:58.914462+0000","last_active":"2024-05-27T12:35:00.965456+0000","last_peered":"2024-05-27T12:35:00.965456+0000","last_clean":"2024-05-27T12:35:00.965456+0000","last_became_active":"2024-05-27T12:34:58.914188+0000","last_became_peered":"2024-05-27T12:34:58.914188+0000","last_unstale":"2024-05-27T12:35:00.965456+0000","last_undegraded":"2024-05-27T12:35:00.965456+0000","last_fullsized":"2024-05-27T12:35:00.965456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983335+0000","last_change":"2024-05-27T12:34:59.952226+0000","last_active":"2024-05-27T12:35:00.983335+0000","last_peered":"2024-05-27T12:35:00.983335+0000","last_clean":"2024-05-27T12:35:00.983335+0000","last_became_active":"2024-05-27T12:34:59.951936+0000","last_became_peered":"2024-05-27T12:34:59.951936+0000","last_unstale":"2024-05-27T12:35:00.983335+0000","last_undegraded":"2024-05-27T12:35:00.983335+0000","last_fullsized":"2024-05-27T12:35:00.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945936+0000","last_change":"2024-05-27T12:34:58.919780+0000","last_active":"2024-05-27T12:35:00.945936+0000","last_peered":"2024-05-27T12:35:00.945936+0000","last_clean":"2024-05-27T12:35:00.945936+0000","last_became_active":"2024-05-27T12:34:58.919505+0000","last_became_peered":"2024-05-27T12:34:58.919505+0000","last_unstale":"2024-05-27T12:35:00.945936+0000","last_undegraded":"2024-05-27T12:35:00.945936+0000","last_fullsized":"2024-05-27T12:35:00.945936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984200+0000","last_change":"2024-05-27T12:34:59.946900+0000","last_active":"2024-05-27T12:35:00.984200+0000","last_peered":"2024-05-27T12:35:00.984200+0000","last_clean":"2024-05-27T12:35:00.984200+0000","last_became_active":"2024-05-27T12:34:59.945045+0000","last_became_peered":"2024-05-27T12:34:59.945045+0000","last_unstale":"2024-05-27T12:35:00.984200+0000","last_undegraded":"2024-05-27T12:35:00.984200+0000","last_fullsized":"2024-05-27T12:35:00.984200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981186+0000","last_change":"2024-05-27T12:34:58.920871+0000","last_active":"2024-05-27T12:35:00.981186+0000","last_peered":"2024-05-27T12:35:00.981186+0000","last_clean":"2024-05-27T12:35:00.981186+0000","last_became_active":"2024-05-27T12:34:58.917367+0000","last_became_peered":"2024-05-27T12:34:58.917367+0000","last_unstale":"2024-05-27T12:35:00.981186+0000","last_undegraded":"2024-05-27T12:35:00.981186+0000","last_fullsized":"2024-05-27T12:35:00.981186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946167+0000","last_change":"2024-05-27T12:34:59.939621+0000","last_active":"2024-05-27T12:35:00.946167+0000","last_peered":"2024-05-27T12:35:00.946167+0000","last_clean":"2024-05-27T12:35:00.946167+0000","last_became_active":"2024-05-27T12:34:59.939269+0000","last_became_peered":"2024-05-27T12:34:59.939269+0000","last_unstale":"2024-05-27T12:35:00.946167+0000","last_undegraded":"2024-05-27T12:35:00.946167+0000","last_fullsized":"2024-05-27T12:35:00.946167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:45.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944012+0000","last_change":"2024-05-27T12:34:58.922915+0000","last_active":"2024-05-27T12:35:00.944012+0000","last_peered":"2024-05-27T12:35:00.944012+0000","last_clean":"2024-05-27T12:35:00.944012+0000","last_became_active":"2024-05-27T12:34:58.922438+0000","last_became_peered":"2024-05-27T12:34:58.922438+0000","last_unstale":"2024-05-27T12:35:00.944012+0000","last_undegraded":"2024-05-27T12:35:00.944012+0000","last_fullsized":"2024-05-27T12:35:00.944012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980353+0000","last_change":"2024-05-27T12:34:59.947355+0000","last_active":"2024-05-27T12:35:00.980353+0000","last_peered":"2024-05-27T12:35:00.980353+0000","last_clean":"2024-05-27T12:35:00.980353+0000","last_became_active":"2024-05-27T12:34:59.947069+0000","last_became_peered":"2024-05-27T12:34:59.947069+0000","last_unstale":"2024-05-27T12:35:00.980353+0000","last_undegraded":"2024-05-27T12:35:00.980353+0000","last_fullsized":"2024-05-27T12:35:00.980353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943707+0000","last_change":"2024-05-27T12:34:58.911817+0000","last_active":"2024-05-27T12:35:00.943707+0000","last_peered":"2024-05-27T12:35:00.943707+0000","last_clean":"2024-05-27T12:35:00.943707+0000","last_became_active":"2024-05-27T12:34:58.911528+0000","last_became_peered":"2024-05-27T12:34:58.911528+0000","last_unstale":"2024-05-27T12:35:00.943707+0000","last_undegraded":"2024-05-27T12:35:00.943707+0000","last_fullsized":"2024-05-27T12:35:00.943707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981479+0000","last_change":"2024-05-27T12:34:59.944469+0000","last_active":"2024-05-27T12:35:00.981479+0000","last_peered":"2024-05-27T12:35:00.981479+0000","last_clean":"2024-05-27T12:35:00.981479+0000","last_became_active":"2024-05-27T12:34:59.943952+0000","last_became_peered":"2024-05-27T12:34:59.943952+0000","last_unstale":"2024-05-27T12:35:00.981479+0000","last_undegraded":"2024-05-27T12:35:00.981479+0000","last_fullsized":"2024-05-27T12:35:00.981479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982842+0000","last_change":"2024-05-27T12:34:58.928167+0000","last_active":"2024-05-27T12:35:00.982842+0000","last_peered":"2024-05-27T12:35:00.982842+0000","last_clean":"2024-05-27T12:35:00.982842+0000","last_became_active":"2024-05-27T12:34:58.927922+0000","last_became_peered":"2024-05-27T12:34:58.927922+0000","last_unstale":"2024-05-27T12:35:00.982842+0000","last_undegraded":"2024-05-27T12:35:00.982842+0000","last_fullsized":"2024-05-27T12:35:00.982842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:03.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945600+0000","last_change":"2024-05-27T12:34:59.951341+0000","last_active":"2024-05-27T12:35:00.945600+0000","last_peered":"2024-05-27T12:35:00.945600+0000","last_clean":"2024-05-27T12:35:00.945600+0000","last_became_active":"2024-05-27T12:34:59.949217+0000","last_became_peered":"2024-05-27T12:34:59.949217+0000","last_unstale":"2024-05-27T12:35:00.945600+0000","last_undegraded":"2024-05-27T12:35:00.945600+0000","last_fullsized":"2024-05-27T12:35:00.945600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980544+0000","last_change":"2024-05-27T12:34:59.954080+0000","last_active":"2024-05-27T12:35:00.980544+0000","last_peered":"2024-05-27T12:35:00.980544+0000","last_clean":"2024-05-27T12:35:00.980544+0000","last_became_active":"2024-05-27T12:34:59.946377+0000","last_became_peered":"2024-05-27T12:34:59.946377+0000","last_unstale":"2024-05-27T12:35:00.980544+0000","last_undegraded":"2024-05-27T12:35:00.980544+0000","last_fullsized":"2024-05-27T12:35:00.980544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982176+0000","last_change":"2024-05-27T12:34:58.926815+0000","last_active":"2024-05-27T12:35:00.982176+0000","last_peered":"2024-05-27T12:35:00.982176+0000","last_clean":"2024-05-27T12:35:00.982176+0000","last_became_active":"2024-05-27T12:34:58.926419+0000","last_became_peered":"2024-05-27T12:34:58.926419+0000","last_unstale":"2024-05-27T12:35:00.982176+0000","last_undegraded":"2024-05-27T12:35:00.982176+0000","last_fullsized":"2024-05-27T12:35:00.982176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:05.278200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980205+0000","last_change":"2024-05-27T12:34:59.942376+0000","last_active":"2024-05-27T12:35:00.980205+0000","last_peered":"2024-05-27T12:35:00.980205+0000","last_clean":"2024-05-27T12:35:00.980205+0000","last_became_active":"2024-05-27T12:34:59.941083+0000","last_became_peered":"2024-05-27T12:34:59.941083+0000","last_unstale":"2024-05-27T12:35:00.980205+0000","last_undegraded":"2024-05-27T12:35:00.980205+0000","last_fullsized":"2024-05-27T12:35:00.980205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982827+0000","last_change":"2024-05-27T12:34:58.927394+0000","last_active":"2024-05-27T12:35:00.982827+0000","last_peered":"2024-05-27T12:35:00.982827+0000","last_clean":"2024-05-27T12:35:00.982827+0000","last_became_active":"2024-05-27T12:34:58.927124+0000","last_became_peered":"2024-05-27T12:34:58.927124+0000","last_unstale":"2024-05-27T12:35:00.982827+0000","last_undegraded":"2024-05-27T12:35:00.982827+0000","last_fullsized":"2024-05-27T12:35:00.982827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:53.128806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984386+0000","last_change":"2024-05-27T12:34:59.946923+0000","last_active":"2024-05-27T12:35:00.984386+0000","last_peered":"2024-05-27T12:35:00.984386+0000","last_clean":"2024-05-27T12:35:00.984386+0000","last_became_active":"2024-05-27T12:34:59.946554+0000","last_became_peered":"2024-05-27T12:34:59.946554+0000","last_unstale":"2024-05-27T12:35:00.984386+0000","last_undegraded":"2024-05-27T12:35:00.984386+0000","last_fullsized":"2024-05-27T12:35:00.984386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981709+0000","last_change":"2024-05-27T12:34:58.912889+0000","last_active":"2024-05-27T12:35:00.981709+0000","last_peered":"2024-05-27T12:35:00.981709+0000","last_clean":"2024-05-27T12:35:00.981709+0000","last_became_active":"2024-05-27T12:34:58.912617+0000","last_became_peered":"2024-05-27T12:34:58.912617+0000","last_unstale":"2024-05-27T12:35:00.981709+0000","last_undegraded":"2024-05-27T12:35:00.981709+0000","last_fullsized":"2024-05-27T12:35:00.981709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981440+0000","last_change":"2024-05-27T12:34:59.941269+0000","last_active":"2024-05-27T12:35:00.981440+0000","last_peered":"2024-05-27T12:35:00.981440+0000","last_clean":"2024-05-27T12:35:00.981440+0000","last_became_active":"2024-05-27T12:34:59.941004+0000","last_became_peered":"2024-05-27T12:34:59.941004+0000","last_unstale":"2024-05-27T12:35:00.981440+0000","last_undegraded":"2024-05-27T12:35:00.981440+0000","last_fullsized":"2024-05-27T12:35:00.981440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984278+0000","last_change":"2024-05-27T12:34:58.918917+0000","last_active":"2024-05-27T12:35:00.984278+0000","last_peered":"2024-05-27T12:35:00.984278+0000","last_clean":"2024-05-27T12:35:00.984278+0000","last_became_active":"2024-05-27T12:34:58.918499+0000","last_became_peered":"2024-05-27T12:34:58.918499+0000","last_unstale":"2024-05-27T12:35:00.984278+0000","last_undegraded":"2024-05-27T12:35:00.984278+0000","last_fullsized":"2024-05-27T12:35:00.984278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:35:04.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:35:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.580+0000 7fda62800640 1 Processor -- start 2024-05-27T12:35:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.580+0000 7fda62800640 1 -- start start 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.581+0000 7fda62800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fda5c15ea30 0x7fda5c0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.581+0000 7fda62800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.581+0000 7fda62800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c15ee10 con 0x7fda5c0b4510 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.581+0000 7fda62800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fda5c0d2830 con 0x7fda5c15ea30 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.581+0000 7fda62800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c0d2a60 con 0x7fda5c0d4ae0 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fda5c15ea30 0x7fda5c0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fda5c15ea30 0x7fda5c0d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56396/0 (socket says 172.21.15.94:56396) 2024-05-27T12:35:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 -- 172.21.15.94:0/67117733 learned_addr learned my addr 172.21.15.94:0/67117733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:04.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 -- 172.21.15.94:0/67117733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 msgr2=0x7fda5c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:04.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61e00640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 -- 172.21.15.94:0/67117733 >> v1:172.21.15.97:6789/0 conn(0x7fda5c0b4510 legacy=0x7fda5c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:04.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.582+0000 7fda61400640 1 -- 172.21.15.94:0/67117733 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c0d2d40 con 0x7fda5c15ea30 2024-05-27T12:35:04.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.583+0000 7fda61400640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ea30 0x7fda5c0d22f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d39285152291457 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.583+0000 7fda5be00640 1 -- 172.21.15.94:0/67117733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50037070 con 0x7fda5c15ea30 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.583+0000 7fda5be00640 1 -- 172.21.15.94:0/67117733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda50032440 con 0x7fda5c15ea30 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda5be00640 1 -- 172.21.15.94:0/67117733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50032810 con 0x7fda5c15ea30 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 -- 172.21.15.94:0/67117733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ea30 msgr2=0x7fda5c0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ea30 0x7fda5c0d22f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5002a2a0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 -- 172.21.15.94:0/67117733 shutdown_connections 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 --2- 172.21.15.94:0/67117733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c15ea30 0x7fda5c0d22f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 -- 172.21.15.94:0/67117733 >> 172.21.15.94:0/67117733 conn(0x7fda5c09da70 msgr2=0x7fda5c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 -- 172.21.15.94:0/67117733 shutdown_connections 2024-05-27T12:35:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.584+0000 7fda62800640 1 -- 172.21.15.94:0/67117733 wait complete. 2024-05-27T12:35:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.586+0000 7fda62800640 1 Processor -- start 2024-05-27T12:35:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.586+0000 7fda62800640 1 -- start start 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c15ea30 0x7fda5c0c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c0d35e0 con 0x7fda5c0b4510 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c0d3300 con 0x7fda5c15ea30 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda62800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c0cfb50 con 0x7fda5c0d4ae0 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda61400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda61400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56408/0 (socket says 172.21.15.94:56408) 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.587+0000 7fda61400640 1 -- 172.21.15.94:0/3200692543 learned_addr learned my addr 172.21.15.94:0/3200692543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61e00640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c15ea30 0x7fda5c0c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 -- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 msgr2=0x7fda5c0b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 -- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c15ea30 msgr2=0x7fda5c0c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c15ea30 0x7fda5c0c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 -- 172.21.15.94:0/3200692543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c1541f0 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda61400640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50041020 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda50030970 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c0b9040 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50037040 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.588+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c0b9600 con 0x7fda5c0b4510 2024-05-27T12:35:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.590+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c075460 con 0x7fda5c0b4510 2024-05-27T12:35:04.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.590+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda50044020 con 0x7fda5c0b4510 2024-05-27T12:35:04.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.592+0000 7fda59600640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:04.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.592+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda500ac570 con 0x7fda5c0b4510 2024-05-27T12:35:04.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.593+0000 7fda60a00640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.594+0000 7fda60a00640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 0x7fda3c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0d54c0 tx=0x7fda48007a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:04.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.596+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda5004b030 con 0x7fda5c0b4510 2024-05-27T12:35:04.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.792+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda5c0b13d0 con 0x7fda3c06c8f0 2024-05-27T12:35:04.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.810+0000 7fda59600640 1 -- 172.21.15.94:0/3200692543 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7fda48055840 con 0x7fda3c06c8f0 2024-05-27T12:35:04.813 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:04.814 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 msgr2=0x7fda3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 0x7fda3c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0d54c0 tx=0x7fda48007a20 comp rx=0 tx=0).stop 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 msgr2=0x7fda5c0b8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda500046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 shutdown_connections 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fda3c06c8f0 0x7fda3c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5c15ea30 0x7fda5c0c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5c0d4ae0 0x7fda5c0b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 --2- 172.21.15.94:0/3200692543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c0b4510 0x7fda5c0b8440 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:04.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 >> 172.21.15.94:0/3200692543 conn(0x7fda5c09da70 msgr2=0x7fda5c0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:04.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 shutdown_connections 2024-05-27T12:35:04.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:04.816+0000 7fda62800640 1 -- 172.21.15.94:0/3200692543 wait complete. 2024-05-27T12:35:04.835 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-27T12:35:02.859161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.983202"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982878+0000","last_change":"2024-05-27T12:34:59.958231+0000","last_active":"2024-05-27T12:35:00.982878+0000","last_peered":"2024-05-27T12:35:00.982878+0000","last_clean":"2024-05-27T12:35:00.982878+0000","last_became_active":"2024-05-27T12:34:59.957423+0000","last_became_peered":"2024-05-27T12:34:59.957423+0000","last_unstale":"2024-05-27T12:35:00.982878+0000","last_undegraded":"2024-05-27T12:35:00.982878+0000","last_fullsized":"2024-05-27T12:35:00.982878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944746+0000","last_change":"2024-05-27T12:34:59.945994+0000","last_active":"2024-05-27T12:35:00.944746+0000","last_peered":"2024-05-27T12:35:00.944746+0000","last_clean":"2024-05-27T12:35:00.944746+0000","last_became_active":"2024-05-27T12:34:59.945737+0000","last_became_peered":"2024-05-27T12:34:59.945737+0000","last_unstale":"2024-05-27T12:35:00.944746+0000","last_undegraded":"2024-05-27T12:35:00.944746+0000","last_fullsized":"2024-05-27T12:35:00.944746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983788+0000","last_change":"2024-05-27T12:34:59.941922+0000","last_active":"2024-05-27T12:35:00.983788+0000","last_peered":"2024-05-27T12:35:00.983788+0000","last_clean":"2024-05-27T12:35:00.983788+0000","last_became_active":"2024-05-27T12:34:59.941543+0000","last_became_peered":"2024-05-27T12:34:59.941543+0000","last_unstale":"2024-05-27T12:35:00.983788+0000","last_undegraded":"2024-05-27T12:35:00.983788+0000","last_fullsized":"2024-05-27T12:35:00.983788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979987+0000","last_change":"2024-05-27T12:34:59.946464+0000","last_active":"2024-05-27T12:35:00.979987+0000","last_peered":"2024-05-27T12:35:00.979987+0000","last_clean":"2024-05-27T12:35:00.979987+0000","last_became_active":"2024-05-27T12:34:59.943307+0000","last_became_peered":"2024-05-27T12:34:59.943307+0000","last_unstale":"2024-05-27T12:35:00.979987+0000","last_undegraded":"2024-05-27T12:35:00.979987+0000","last_fullsized":"2024-05-27T12:35:00.979987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982559+0000","last_change":"2024-05-27T12:34:59.953536+0000","last_active":"2024-05-27T12:35:00.982559+0000","last_peered":"2024-05-27T12:35:00.982559+0000","last_clean":"2024-05-27T12:35:00.982559+0000","last_became_active":"2024-05-27T12:34:59.953266+0000","last_became_peered":"2024-05-27T12:34:59.953266+0000","last_unstale":"2024-05-27T12:35:00.982559+0000","last_undegraded":"2024-05-27T12:35:00.982559+0000","last_fullsized":"2024-05-27T12:35:00.982559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979484+0000","last_change":"2024-05-27T12:34:59.939835+0000","last_active":"2024-05-27T12:35:00.979484+0000","last_peered":"2024-05-27T12:35:00.979484+0000","last_clean":"2024-05-27T12:35:00.979484+0000","last_became_active":"2024-05-27T12:34:59.938363+0000","last_became_peered":"2024-05-27T12:34:59.938363+0000","last_unstale":"2024-05-27T12:35:00.979484+0000","last_undegraded":"2024-05-27T12:35:00.979484+0000","last_fullsized":"2024-05-27T12:35:00.979484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981481+0000","last_change":"2024-05-27T12:34:59.949943+0000","last_active":"2024-05-27T12:35:00.981481+0000","last_peered":"2024-05-27T12:35:00.981481+0000","last_clean":"2024-05-27T12:35:00.981481+0000","last_became_active":"2024-05-27T12:34:59.949473+0000","last_became_peered":"2024-05-27T12:34:59.949473+0000","last_unstale":"2024-05-27T12:35:00.981481+0000","last_undegraded":"2024-05-27T12:35:00.981481+0000","last_fullsized":"2024-05-27T12:35:00.981481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980190+0000","last_change":"2024-05-27T12:34:59.954206+0000","last_active":"2024-05-27T12:35:00.980190+0000","last_peered":"2024-05-27T12:35:00.980190+0000","last_clean":"2024-05-27T12:35:00.980190+0000","last_became_active":"2024-05-27T12:34:59.953560+0000","last_became_peered":"2024-05-27T12:34:59.953560+0000","last_unstale":"2024-05-27T12:35:00.980190+0000","last_undegraded":"2024-05-27T12:35:00.980190+0000","last_fullsized":"2024-05-27T12:35:00.980190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982883+0000","last_change":"2024-05-27T12:34:59.960424+0000","last_active":"2024-05-27T12:35:00.982883+0000","last_peered":"2024-05-27T12:35:00.982883+0000","last_clean":"2024-05-27T12:35:00.982883+0000","last_became_active":"2024-05-27T12:34:59.959909+0000","last_became_peered":"2024-05-27T12:34:59.959909+0000","last_unstale":"2024-05-27T12:35:00.982883+0000","last_undegraded":"2024-05-27T12:35:00.982883+0000","last_fullsized":"2024-05-27T12:35:00.982883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945456+0000","last_change":"2024-05-27T12:34:59.953459+0000","last_active":"2024-05-27T12:35:00.945456+0000","last_peered":"2024-05-27T12:35:00.945456+0000","last_clean":"2024-05-27T12:35:00.945456+0000","last_became_active":"2024-05-27T12:34:59.947619+0000","last_became_peered":"2024-05-27T12:34:59.947619+0000","last_unstale":"2024-05-27T12:35:00.945456+0000","last_undegraded":"2024-05-27T12:35:00.945456+0000","last_fullsized":"2024-05-27T12:35:00.945456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945452+0000","last_change":"2024-05-27T12:34:59.956011+0000","last_active":"2024-05-27T12:35:00.945452+0000","last_peered":"2024-05-27T12:35:00.945452+0000","last_clean":"2024-05-27T12:35:00.945452+0000","last_became_active":"2024-05-27T12:34:59.955642+0000","last_became_peered":"2024-05-27T12:34:59.955642+0000","last_unstale":"2024-05-27T12:35:00.945452+0000","last_undegraded":"2024-05-27T12:35:00.945452+0000","last_fullsized":"2024-05-27T12:35:00.945452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947001+0000","last_change":"2024-05-27T12:34:59.948810+0000","last_active":"2024-05-27T12:35:00.947001+0000","last_peered":"2024-05-27T12:35:00.947001+0000","last_clean":"2024-05-27T12:35:00.947001+0000","last_became_active":"2024-05-27T12:34:59.948353+0000","last_became_peered":"2024-05-27T12:34:59.948353+0000","last_unstale":"2024-05-27T12:35:00.947001+0000","last_undegraded":"2024-05-27T12:35:00.947001+0000","last_fullsized":"2024-05-27T12:35:00.947001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:52.789382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981338+0000","last_change":"2024-05-27T12:34:59.944238+0000","last_active":"2024-05-27T12:35:00.981338+0000","last_peered":"2024-05-27T12:35:00.981338+0000","last_clean":"2024-05-27T12:35:00.981338+0000","last_became_active":"2024-05-27T12:34:59.943490+0000","last_became_peered":"2024-05-27T12:34:59.943490+0000","last_unstale":"2024-05-27T12:35:00.981338+0000","last_undegraded":"2024-05-27T12:35:00.981338+0000","last_fullsized":"2024-05-27T12:35:00.981338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983782+0000","last_change":"2024-05-27T12:34:59.956775+0000","last_active":"2024-05-27T12:35:00.983782+0000","last_peered":"2024-05-27T12:35:00.983782+0000","last_clean":"2024-05-27T12:35:00.983782+0000","last_became_active":"2024-05-27T12:34:59.954558+0000","last_became_peered":"2024-05-27T12:34:59.954558+0000","last_unstale":"2024-05-27T12:35:00.983782+0000","last_undegraded":"2024-05-27T12:35:00.983782+0000","last_fullsized":"2024-05-27T12:35:00.983782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944437+0000","last_change":"2024-05-27T12:34:59.952264+0000","last_active":"2024-05-27T12:35:00.944437+0000","last_peered":"2024-05-27T12:35:00.944437+0000","last_clean":"2024-05-27T12:35:00.944437+0000","last_became_active":"2024-05-27T12:34:59.951960+0000","last_became_peered":"2024-05-27T12:34:59.951960+0000","last_unstale":"2024-05-27T12:35:00.944437+0000","last_undegraded":"2024-05-27T12:35:00.944437+0000","last_fullsized":"2024-05-27T12:35:00.944437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979305+0000","last_change":"2024-05-27T12:34:59.936327+0000","last_active":"2024-05-27T12:35:00.979305+0000","last_peered":"2024-05-27T12:35:00.979305+0000","last_clean":"2024-05-27T12:35:00.979305+0000","last_became_active":"2024-05-27T12:34:59.935576+0000","last_became_peered":"2024-05-27T12:34:59.935576+0000","last_unstale":"2024-05-27T12:35:00.979305+0000","last_undegraded":"2024-05-27T12:35:00.979305+0000","last_fullsized":"2024-05-27T12:35:00.979305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980066+0000","last_change":"2024-05-27T12:34:59.956063+0000","last_active":"2024-05-27T12:35:00.980066+0000","last_peered":"2024-05-27T12:35:00.980066+0000","last_clean":"2024-05-27T12:35:00.980066+0000","last_became_active":"2024-05-27T12:34:59.955763+0000","last_became_peered":"2024-05-27T12:34:59.955763+0000","last_unstale":"2024-05-27T12:35:00.980066+0000","last_undegraded":"2024-05-27T12:35:00.980066+0000","last_fullsized":"2024-05-27T12:35:00.980066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981742+0000","last_change":"2024-05-27T12:34:59.937118+0000","last_active":"2024-05-27T12:35:00.981742+0000","last_peered":"2024-05-27T12:35:00.981742+0000","last_clean":"2024-05-27T12:35:00.981742+0000","last_became_active":"2024-05-27T12:34:59.936814+0000","last_became_peered":"2024-05-27T12:34:59.936814+0000","last_unstale":"2024-05-27T12:35:00.981742+0000","last_undegraded":"2024-05-27T12:35:00.981742+0000","last_fullsized":"2024-05-27T12:35:00.981742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:54.133638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983099+0000","last_change":"2024-05-27T12:34:59.952246+0000","last_active":"2024-05-27T12:35:00.983099+0000","last_peered":"2024-05-27T12:35:00.983099+0000","last_clean":"2024-05-27T12:35:00.983099+0000","last_became_active":"2024-05-27T12:34:59.951939+0000","last_became_peered":"2024-05-27T12:34:59.951939+0000","last_unstale":"2024-05-27T12:35:00.983099+0000","last_undegraded":"2024-05-27T12:35:00.983099+0000","last_fullsized":"2024-05-27T12:35:00.983099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945011+0000","last_change":"2024-05-27T12:34:59.947085+0000","last_active":"2024-05-27T12:35:00.945011+0000","last_peered":"2024-05-27T12:35:00.945011+0000","last_clean":"2024-05-27T12:35:00.945011+0000","last_became_active":"2024-05-27T12:34:59.946740+0000","last_became_peered":"2024-05-27T12:34:59.946740+0000","last_unstale":"2024-05-27T12:35:00.945011+0000","last_undegraded":"2024-05-27T12:35:00.945011+0000","last_fullsized":"2024-05-27T12:35:00.945011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982971+0000","last_change":"2024-05-27T12:34:59.956268+0000","last_active":"2024-05-27T12:35:00.982971+0000","last_peered":"2024-05-27T12:35:00.982971+0000","last_clean":"2024-05-27T12:35:00.982971+0000","last_became_active":"2024-05-27T12:34:59.950964+0000","last_became_peered":"2024-05-27T12:34:59.950964+0000","last_unstale":"2024-05-27T12:35:00.982971+0000","last_undegraded":"2024-05-27T12:35:00.982971+0000","last_fullsized":"2024-05-27T12:35:00.982971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:12.683859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979683+0000","last_change":"2024-05-27T12:34:59.956010+0000","last_active":"2024-05-27T12:35:00.979683+0000","last_peered":"2024-05-27T12:35:00.979683+0000","last_clean":"2024-05-27T12:35:00.979683+0000","last_became_active":"2024-05-27T12:34:59.955692+0000","last_became_peered":"2024-05-27T12:34:59.955692+0000","last_unstale":"2024-05-27T12:35:00.979683+0000","last_undegraded":"2024-05-27T12:35:00.979683+0000","last_fullsized":"2024-05-27T12:35:00.979683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980587+0000","last_change":"2024-05-27T12:34:59.950671+0000","last_active":"2024-05-27T12:35:00.980587+0000","last_peered":"2024-05-27T12:35:00.980587+0000","last_clean":"2024-05-27T12:35:00.980587+0000","last_became_active":"2024-05-27T12:34:59.950168+0000","last_became_peered":"2024-05-27T12:34:59.950168+0000","last_unstale":"2024-05-27T12:35:00.980587+0000","last_undegraded":"2024-05-27T12:35:00.980587+0000","last_fullsized":"2024-05-27T12:35:00.980587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979458+0000","last_change":"2024-05-27T12:34:59.947016+0000","last_active":"2024-05-27T12:35:00.979458+0000","last_peered":"2024-05-27T12:35:00.979458+0000","last_clean":"2024-05-27T12:35:00.979458+0000","last_became_active":"2024-05-27T12:34:59.946710+0000","last_became_peered":"2024-05-27T12:34:59.946710+0000","last_unstale":"2024-05-27T12:35:00.979458+0000","last_undegraded":"2024-05-27T12:35:00.979458+0000","last_fullsized":"2024-05-27T12:35:00.979458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982935+0000","last_change":"2024-05-27T12:34:59.952637+0000","last_active":"2024-05-27T12:35:00.982935+0000","last_peered":"2024-05-27T12:35:00.982935+0000","last_clean":"2024-05-27T12:35:00.982935+0000","last_became_active":"2024-05-27T12:34:59.951346+0000","last_became_peered":"2024-05-27T12:34:59.951346+0000","last_unstale":"2024-05-27T12:35:00.982935+0000","last_undegraded":"2024-05-27T12:35:00.982935+0000","last_fullsized":"2024-05-27T12:35:00.982935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945571+0000","last_change":"2024-05-27T12:34:59.958301+0000","last_active":"2024-05-27T12:35:00.945571+0000","last_peered":"2024-05-27T12:35:00.945571+0000","last_clean":"2024-05-27T12:35:00.945571+0000","last_became_active":"2024-05-27T12:34:59.957996+0000","last_became_peered":"2024-05-27T12:34:59.957996+0000","last_unstale":"2024-05-27T12:35:00.945571+0000","last_undegraded":"2024-05-27T12:35:00.945571+0000","last_fullsized":"2024-05-27T12:35:00.945571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944292+0000","last_change":"2024-05-27T12:34:59.949515+0000","last_active":"2024-05-27T12:35:00.944292+0000","last_peered":"2024-05-27T12:35:00.944292+0000","last_clean":"2024-05-27T12:35:00.944292+0000","last_became_active":"2024-05-27T12:34:59.945960+0000","last_became_peered":"2024-05-27T12:34:59.945960+0000","last_unstale":"2024-05-27T12:35:00.944292+0000","last_undegraded":"2024-05-27T12:35:00.944292+0000","last_fullsized":"2024-05-27T12:35:00.944292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980945+0000","last_change":"2024-05-27T12:34:59.950471+0000","last_active":"2024-05-27T12:35:00.980945+0000","last_peered":"2024-05-27T12:35:00.980945+0000","last_clean":"2024-05-27T12:35:00.980945+0000","last_became_active":"2024-05-27T12:34:59.950135+0000","last_became_peered":"2024-05-27T12:34:59.950135+0000","last_unstale":"2024-05-27T12:35:00.980945+0000","last_undegraded":"2024-05-27T12:35:00.980945+0000","last_fullsized":"2024-05-27T12:35:00.980945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983332+0000","last_change":"2024-05-27T12:34:59.942121+0000","last_active":"2024-05-27T12:35:00.983332+0000","last_peered":"2024-05-27T12:35:00.983332+0000","last_clean":"2024-05-27T12:35:00.983332+0000","last_became_active":"2024-05-27T12:34:59.941006+0000","last_became_peered":"2024-05-27T12:34:59.941006+0000","last_unstale":"2024-05-27T12:35:00.983332+0000","last_undegraded":"2024-05-27T12:35:00.983332+0000","last_fullsized":"2024-05-27T12:35:00.983332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979945+0000","last_change":"2024-05-27T12:34:59.957178+0000","last_active":"2024-05-27T12:35:00.979945+0000","last_peered":"2024-05-27T12:35:00.979945+0000","last_clean":"2024-05-27T12:35:00.979945+0000","last_became_active":"2024-05-27T12:34:59.956854+0000","last_became_peered":"2024-05-27T12:34:59.956854+0000","last_unstale":"2024-05-27T12:35:00.979945+0000","last_undegraded":"2024-05-27T12:35:00.979945+0000","last_fullsized":"2024-05-27T12:35:00.979945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980174+0000","last_change":"2024-05-27T12:34:59.954564+0000","last_active":"2024-05-27T12:35:00.980174+0000","last_peered":"2024-05-27T12:35:00.980174+0000","last_clean":"2024-05-27T12:35:00.980174+0000","last_became_active":"2024-05-27T12:34:59.953442+0000","last_became_peered":"2024-05-27T12:34:59.953442+0000","last_unstale":"2024-05-27T12:35:00.980174+0000","last_undegraded":"2024-05-27T12:35:00.980174+0000","last_fullsized":"2024-05-27T12:35:00.980174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946038+0000","last_change":"2024-05-27T12:34:59.936968+0000","last_active":"2024-05-27T12:35:00.946038+0000","last_peered":"2024-05-27T12:35:00.946038+0000","last_clean":"2024-05-27T12:35:00.946038+0000","last_became_active":"2024-05-27T12:34:59.935850+0000","last_became_peered":"2024-05-27T12:34:59.935850+0000","last_unstale":"2024-05-27T12:35:00.946038+0000","last_undegraded":"2024-05-27T12:35:00.946038+0000","last_fullsized":"2024-05-27T12:35:00.946038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947845+0000","last_change":"2024-05-27T12:34:59.955463+0000","last_active":"2024-05-27T12:35:00.947845+0000","last_peered":"2024-05-27T12:35:00.947845+0000","last_clean":"2024-05-27T12:35:00.947845+0000","last_became_active":"2024-05-27T12:34:59.952053+0000","last_became_peered":"2024-05-27T12:34:59.952053+0000","last_unstale":"2024-05-27T12:35:00.947845+0000","last_undegraded":"2024-05-27T12:35:00.947845+0000","last_fullsized":"2024-05-27T12:35:00.947845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981586+0000","last_change":"2024-05-27T12:34:58.926344+0000","last_active":"2024-05-27T12:35:00.981586+0000","last_peered":"2024-05-27T12:35:00.981586+0000","last_clean":"2024-05-27T12:35:00.981586+0000","last_became_active":"2024-05-27T12:34:58.926021+0000","last_became_peered":"2024-05-27T12:34:58.926021+0000","last_unstale":"2024-05-27T12:35:00.981586+0000","last_undegraded":"2024-05-27T12:35:00.981586+0000","last_fullsized":"2024-05-27T12:35:00.981586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:35:59.130525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983155+0000","last_change":"2024-05-27T12:34:59.948432+0000","last_active":"2024-05-27T12:35:00.983155+0000","last_peered":"2024-05-27T12:35:00.983155+0000","last_clean":"2024-05-27T12:35:00.983155+0000","last_became_active":"2024-05-27T12:34:59.946912+0000","last_became_peered":"2024-05-27T12:34:59.946912+0000","last_unstale":"2024-05-27T12:35:00.983155+0000","last_undegraded":"2024-05-27T12:35:00.983155+0000","last_fullsized":"2024-05-27T12:35:00.983155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945458+0000","last_change":"2024-05-27T12:34:58.918241+0000","last_active":"2024-05-27T12:35:00.945458+0000","last_peered":"2024-05-27T12:35:00.945458+0000","last_clean":"2024-05-27T12:35:00.945458+0000","last_became_active":"2024-05-27T12:34:58.917640+0000","last_became_peered":"2024-05-27T12:34:58.917640+0000","last_unstale":"2024-05-27T12:35:00.945458+0000","last_undegraded":"2024-05-27T12:35:00.945458+0000","last_fullsized":"2024-05-27T12:35:00.945458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980740+0000","last_change":"2024-05-27T12:34:58.912568+0000","last_active":"2024-05-27T12:35:00.980740+0000","last_peered":"2024-05-27T12:35:00.980740+0000","last_clean":"2024-05-27T12:35:00.980740+0000","last_became_active":"2024-05-27T12:34:58.912280+0000","last_became_peered":"2024-05-27T12:34:58.912280+0000","last_unstale":"2024-05-27T12:35:00.980740+0000","last_undegraded":"2024-05-27T12:35:00.980740+0000","last_fullsized":"2024-05-27T12:35:00.980740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980543+0000","last_change":"2024-05-27T12:34:59.947954+0000","last_active":"2024-05-27T12:35:00.980543+0000","last_peered":"2024-05-27T12:35:00.980543+0000","last_clean":"2024-05-27T12:35:00.980543+0000","last_became_active":"2024-05-27T12:34:59.947702+0000","last_became_peered":"2024-05-27T12:34:59.947702+0000","last_unstale":"2024-05-27T12:35:00.980543+0000","last_undegraded":"2024-05-27T12:35:00.980543+0000","last_fullsized":"2024-05-27T12:35:00.980543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979570+0000","last_change":"2024-05-27T12:34:58.918329+0000","last_active":"2024-05-27T12:35:00.979570+0000","last_peered":"2024-05-27T12:35:00.979570+0000","last_clean":"2024-05-27T12:35:00.979570+0000","last_became_active":"2024-05-27T12:34:58.918007+0000","last_became_peered":"2024-05-27T12:34:58.918007+0000","last_unstale":"2024-05-27T12:35:00.979570+0000","last_undegraded":"2024-05-27T12:35:00.979570+0000","last_fullsized":"2024-05-27T12:35:00.979570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943753+0000","last_change":"2024-05-27T12:34:59.952134+0000","last_active":"2024-05-27T12:35:00.943753+0000","last_peered":"2024-05-27T12:35:00.943753+0000","last_clean":"2024-05-27T12:35:00.943753+0000","last_became_active":"2024-05-27T12:34:59.951823+0000","last_became_peered":"2024-05-27T12:34:59.951823+0000","last_unstale":"2024-05-27T12:35:00.943753+0000","last_undegraded":"2024-05-27T12:35:00.943753+0000","last_fullsized":"2024-05-27T12:35:00.943753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945424+0000","last_change":"2024-05-27T12:34:58.925614+0000","last_active":"2024-05-27T12:35:00.945424+0000","last_peered":"2024-05-27T12:35:00.945424+0000","last_clean":"2024-05-27T12:35:00.945424+0000","last_became_active":"2024-05-27T12:34:58.925312+0000","last_became_peered":"2024-05-27T12:34:58.925312+0000","last_unstale":"2024-05-27T12:35:00.945424+0000","last_undegraded":"2024-05-27T12:35:00.945424+0000","last_fullsized":"2024-05-27T12:35:00.945424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979670+0000","last_change":"2024-05-27T12:34:59.958727+0000","last_active":"2024-05-27T12:35:00.979670+0000","last_peered":"2024-05-27T12:35:00.979670+0000","last_clean":"2024-05-27T12:35:00.979670+0000","last_became_active":"2024-05-27T12:34:59.958405+0000","last_became_peered":"2024-05-27T12:34:59.958405+0000","last_unstale":"2024-05-27T12:35:00.979670+0000","last_undegraded":"2024-05-27T12:35:00.979670+0000","last_fullsized":"2024-05-27T12:35:00.979670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979703+0000","last_change":"2024-05-27T12:34:58.914733+0000","last_active":"2024-05-27T12:35:00.979703+0000","last_peered":"2024-05-27T12:35:00.979703+0000","last_clean":"2024-05-27T12:35:00.979703+0000","last_became_active":"2024-05-27T12:34:58.914357+0000","last_became_peered":"2024-05-27T12:34:58.914357+0000","last_unstale":"2024-05-27T12:35:00.979703+0000","last_undegraded":"2024-05-27T12:35:00.979703+0000","last_fullsized":"2024-05-27T12:35:00.979703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984178+0000","last_change":"2024-05-27T12:34:59.947773+0000","last_active":"2024-05-27T12:35:00.984178+0000","last_peered":"2024-05-27T12:35:00.984178+0000","last_clean":"2024-05-27T12:35:00.984178+0000","last_became_active":"2024-05-27T12:34:59.946577+0000","last_became_peered":"2024-05-27T12:34:59.946577+0000","last_unstale":"2024-05-27T12:35:00.984178+0000","last_undegraded":"2024-05-27T12:35:00.984178+0000","last_fullsized":"2024-05-27T12:35:00.984178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944069+0000","last_change":"2024-05-27T12:34:58.926880+0000","last_active":"2024-05-27T12:35:00.944069+0000","last_peered":"2024-05-27T12:35:00.944069+0000","last_clean":"2024-05-27T12:35:00.944069+0000","last_became_active":"2024-05-27T12:34:58.926621+0000","last_became_peered":"2024-05-27T12:34:58.926621+0000","last_unstale":"2024-05-27T12:35:00.944069+0000","last_undegraded":"2024-05-27T12:35:00.944069+0000","last_fullsized":"2024-05-27T12:35:00.944069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947431+0000","last_change":"2024-05-27T12:34:59.949465+0000","last_active":"2024-05-27T12:35:00.947431+0000","last_peered":"2024-05-27T12:35:00.947431+0000","last_clean":"2024-05-27T12:35:00.947431+0000","last_became_active":"2024-05-27T12:34:59.949190+0000","last_became_peered":"2024-05-27T12:34:59.949190+0000","last_unstale":"2024-05-27T12:35:00.947431+0000","last_undegraded":"2024-05-27T12:35:00.947431+0000","last_fullsized":"2024-05-27T12:35:00.947431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:27.503111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945133+0000","last_change":"2024-05-27T12:34:58.905974+0000","last_active":"2024-05-27T12:35:00.945133+0000","last_peered":"2024-05-27T12:35:00.945133+0000","last_clean":"2024-05-27T12:35:00.945133+0000","last_became_active":"2024-05-27T12:34:58.905596+0000","last_became_peered":"2024-05-27T12:34:58.905596+0000","last_unstale":"2024-05-27T12:35:00.945133+0000","last_undegraded":"2024-05-27T12:35:00.945133+0000","last_fullsized":"2024-05-27T12:35:00.945133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982755+0000","last_change":"2024-05-27T12:34:59.953412+0000","last_active":"2024-05-27T12:35:00.982755+0000","last_peered":"2024-05-27T12:35:00.982755+0000","last_clean":"2024-05-27T12:35:00.982755+0000","last_became_active":"2024-05-27T12:34:59.953103+0000","last_became_peered":"2024-05-27T12:34:59.953103+0000","last_unstale":"2024-05-27T12:35:00.982755+0000","last_undegraded":"2024-05-27T12:35:00.982755+0000","last_fullsized":"2024-05-27T12:35:00.982755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983281+0000","last_change":"2024-05-27T12:34:58.909953+0000","last_active":"2024-05-27T12:35:00.983281+0000","last_peered":"2024-05-27T12:35:00.983281+0000","last_clean":"2024-05-27T12:35:00.983281+0000","last_became_active":"2024-05-27T12:34:58.909633+0000","last_became_peered":"2024-05-27T12:34:58.909633+0000","last_unstale":"2024-05-27T12:35:00.983281+0000","last_undegraded":"2024-05-27T12:35:00.983281+0000","last_fullsized":"2024-05-27T12:35:00.983281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:39.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983591+0000","last_change":"2024-05-27T12:34:59.955304+0000","last_active":"2024-05-27T12:35:00.983591+0000","last_peered":"2024-05-27T12:35:00.983591+0000","last_clean":"2024-05-27T12:35:00.983591+0000","last_became_active":"2024-05-27T12:34:59.954940+0000","last_became_peered":"2024-05-27T12:34:59.954940+0000","last_unstale":"2024-05-27T12:35:00.983591+0000","last_undegraded":"2024-05-27T12:35:00.983591+0000","last_fullsized":"2024-05-27T12:35:00.983591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980018+0000","last_change":"2024-05-27T12:34:58.914660+0000","last_active":"2024-05-27T12:35:00.980018+0000","last_peered":"2024-05-27T12:35:00.980018+0000","last_clean":"2024-05-27T12:35:00.980018+0000","last_became_active":"2024-05-27T12:34:58.914368+0000","last_became_peered":"2024-05-27T12:34:58.914368+0000","last_unstale":"2024-05-27T12:35:00.980018+0000","last_undegraded":"2024-05-27T12:35:00.980018+0000","last_fullsized":"2024-05-27T12:35:00.980018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982688+0000","last_change":"2024-05-27T12:34:59.951438+0000","last_active":"2024-05-27T12:35:00.982688+0000","last_peered":"2024-05-27T12:35:00.982688+0000","last_clean":"2024-05-27T12:35:00.982688+0000","last_became_active":"2024-05-27T12:34:59.951140+0000","last_became_peered":"2024-05-27T12:34:59.951140+0000","last_unstale":"2024-05-27T12:35:00.982688+0000","last_undegraded":"2024-05-27T12:35:00.982688+0000","last_fullsized":"2024-05-27T12:35:00.982688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944792+0000","last_change":"2024-05-27T12:34:58.918066+0000","last_active":"2024-05-27T12:35:00.944792+0000","last_peered":"2024-05-27T12:35:00.944792+0000","last_clean":"2024-05-27T12:35:00.944792+0000","last_became_active":"2024-05-27T12:34:58.917613+0000","last_became_peered":"2024-05-27T12:34:58.917613+0000","last_unstale":"2024-05-27T12:35:00.944792+0000","last_undegraded":"2024-05-27T12:35:00.944792+0000","last_fullsized":"2024-05-27T12:35:00.944792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944830+0000","last_change":"2024-05-27T12:34:59.943832+0000","last_active":"2024-05-27T12:35:00.944830+0000","last_peered":"2024-05-27T12:35:00.944830+0000","last_clean":"2024-05-27T12:35:00.944830+0000","last_became_active":"2024-05-27T12:34:59.943535+0000","last_became_peered":"2024-05-27T12:34:59.943535+0000","last_unstale":"2024-05-27T12:35:00.944830+0000","last_undegraded":"2024-05-27T12:35:00.944830+0000","last_fullsized":"2024-05-27T12:35:00.944830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979766+0000","last_change":"2024-05-27T12:34:58.897545+0000","last_active":"2024-05-27T12:35:00.979766+0000","last_peered":"2024-05-27T12:35:00.979766+0000","last_clean":"2024-05-27T12:35:00.979766+0000","last_became_active":"2024-05-27T12:34:58.895961+0000","last_became_peered":"2024-05-27T12:34:58.895961+0000","last_unstale":"2024-05-27T12:35:00.979766+0000","last_undegraded":"2024-05-27T12:35:00.979766+0000","last_fullsized":"2024-05-27T12:35:00.979766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981686+0000","last_change":"2024-05-27T12:34:59.947365+0000","last_active":"2024-05-27T12:35:00.981686+0000","last_peered":"2024-05-27T12:35:00.981686+0000","last_clean":"2024-05-27T12:35:00.981686+0000","last_became_active":"2024-05-27T12:34:59.947060+0000","last_became_peered":"2024-05-27T12:34:59.947060+0000","last_unstale":"2024-05-27T12:35:00.981686+0000","last_undegraded":"2024-05-27T12:35:00.981686+0000","last_fullsized":"2024-05-27T12:35:00.981686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946688+0000","last_change":"2024-05-27T12:34:58.910200+0000","last_active":"2024-05-27T12:35:00.946688+0000","last_peered":"2024-05-27T12:35:00.946688+0000","last_clean":"2024-05-27T12:35:00.946688+0000","last_became_active":"2024-05-27T12:34:58.909819+0000","last_became_peered":"2024-05-27T12:34:58.909819+0000","last_unstale":"2024-05-27T12:35:00.946688+0000","last_undegraded":"2024-05-27T12:35:00.946688+0000","last_fullsized":"2024-05-27T12:35:00.946688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983250+0000","last_change":"2024-05-27T12:34:59.960324+0000","last_active":"2024-05-27T12:35:00.983250+0000","last_peered":"2024-05-27T12:35:00.983250+0000","last_clean":"2024-05-27T12:35:00.983250+0000","last_became_active":"2024-05-27T12:34:59.958530+0000","last_became_peered":"2024-05-27T12:34:59.958530+0000","last_unstale":"2024-05-27T12:35:00.983250+0000","last_undegraded":"2024-05-27T12:35:00.983250+0000","last_fullsized":"2024-05-27T12:35:00.983250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:10:04.239389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982017+0000","last_change":"2024-05-27T12:34:58.905547+0000","last_active":"2024-05-27T12:35:00.982017+0000","last_peered":"2024-05-27T12:35:00.982017+0000","last_clean":"2024-05-27T12:35:00.982017+0000","last_became_active":"2024-05-27T12:34:58.904281+0000","last_became_peered":"2024-05-27T12:34:58.904281+0000","last_unstale":"2024-05-27T12:35:00.982017+0000","last_undegraded":"2024-05-27T12:35:00.982017+0000","last_fullsized":"2024-05-27T12:35:00.982017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983204+0000","last_change":"2024-05-27T12:34:59.960325+0000","last_active":"2024-05-27T12:35:00.983204+0000","last_peered":"2024-05-27T12:35:00.983204+0000","last_clean":"2024-05-27T12:35:00.983204+0000","last_became_active":"2024-05-27T12:34:59.959764+0000","last_became_peered":"2024-05-27T12:34:59.959764+0000","last_unstale":"2024-05-27T12:35:00.983204+0000","last_undegraded":"2024-05-27T12:35:00.983204+0000","last_fullsized":"2024-05-27T12:35:00.983204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:26:24.248099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944296+0000","last_change":"2024-05-27T12:34:58.916163+0000","last_active":"2024-05-27T12:35:00.944296+0000","last_peered":"2024-05-27T12:35:00.944296+0000","last_clean":"2024-05-27T12:35:00.944296+0000","last_became_active":"2024-05-27T12:34:58.915823+0000","last_became_peered":"2024-05-27T12:34:58.915823+0000","last_unstale":"2024-05-27T12:35:00.944296+0000","last_undegraded":"2024-05-27T12:35:00.944296+0000","last_fullsized":"2024-05-27T12:35:00.944296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980392+0000","last_change":"2024-05-27T12:34:59.944578+0000","last_active":"2024-05-27T12:35:00.980392+0000","last_peered":"2024-05-27T12:35:00.980392+0000","last_clean":"2024-05-27T12:35:00.980392+0000","last_became_active":"2024-05-27T12:34:59.941815+0000","last_became_peered":"2024-05-27T12:34:59.941815+0000","last_unstale":"2024-05-27T12:35:00.980392+0000","last_undegraded":"2024-05-27T12:35:00.980392+0000","last_fullsized":"2024-05-27T12:35:00.980392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980249+0000","last_change":"2024-05-27T12:34:58.921269+0000","last_active":"2024-05-27T12:35:00.980249+0000","last_peered":"2024-05-27T12:35:00.980249+0000","last_clean":"2024-05-27T12:35:00.980249+0000","last_became_active":"2024-05-27T12:34:58.920929+0000","last_became_peered":"2024-05-27T12:34:58.920929+0000","last_unstale":"2024-05-27T12:35:00.980249+0000","last_undegraded":"2024-05-27T12:35:00.980249+0000","last_fullsized":"2024-05-27T12:35:00.980249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982666+0000","last_change":"2024-05-27T12:34:59.959353+0000","last_active":"2024-05-27T12:35:00.982666+0000","last_peered":"2024-05-27T12:35:00.982666+0000","last_clean":"2024-05-27T12:35:00.982666+0000","last_became_active":"2024-05-27T12:34:59.959097+0000","last_became_peered":"2024-05-27T12:34:59.959097+0000","last_unstale":"2024-05-27T12:35:00.982666+0000","last_undegraded":"2024-05-27T12:35:00.982666+0000","last_fullsized":"2024-05-27T12:35:00.982666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981367+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:35:00.981367+0000","last_peered":"2024-05-27T12:35:00.981367+0000","last_clean":"2024-05-27T12:35:00.981367+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:35:00.981367+0000","last_undegraded":"2024-05-27T12:35:00.981367+0000","last_fullsized":"2024-05-27T12:35:00.981367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979291+0000","last_change":"2024-05-27T12:34:58.920919+0000","last_active":"2024-05-27T12:35:00.979291+0000","last_peered":"2024-05-27T12:35:00.979291+0000","last_clean":"2024-05-27T12:35:00.979291+0000","last_became_active":"2024-05-27T12:34:58.918391+0000","last_became_peered":"2024-05-27T12:34:58.918391+0000","last_unstale":"2024-05-27T12:35:00.979291+0000","last_undegraded":"2024-05-27T12:35:00.979291+0000","last_fullsized":"2024-05-27T12:35:00.979291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981108+0000","last_change":"2024-05-27T12:34:59.959595+0000","last_active":"2024-05-27T12:35:00.981108+0000","last_peered":"2024-05-27T12:35:00.981108+0000","last_clean":"2024-05-27T12:35:00.981108+0000","last_became_active":"2024-05-27T12:34:59.959298+0000","last_became_peered":"2024-05-27T12:34:59.959298+0000","last_unstale":"2024-05-27T12:35:00.981108+0000","last_undegraded":"2024-05-27T12:35:00.981108+0000","last_fullsized":"2024-05-27T12:35:00.981108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981337+0000","last_change":"2024-05-27T12:34:58.926438+0000","last_active":"2024-05-27T12:35:00.981337+0000","last_peered":"2024-05-27T12:35:00.981337+0000","last_clean":"2024-05-27T12:35:00.981337+0000","last_became_active":"2024-05-27T12:34:58.926205+0000","last_became_peered":"2024-05-27T12:34:58.926205+0000","last_unstale":"2024-05-27T12:35:00.981337+0000","last_undegraded":"2024-05-27T12:35:00.981337+0000","last_fullsized":"2024-05-27T12:35:00.981337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982896+0000","last_change":"2024-05-27T12:34:59.947056+0000","last_active":"2024-05-27T12:35:00.982896+0000","last_peered":"2024-05-27T12:35:00.982896+0000","last_clean":"2024-05-27T12:35:00.982896+0000","last_became_active":"2024-05-27T12:34:59.946724+0000","last_became_peered":"2024-05-27T12:34:59.946724+0000","last_unstale":"2024-05-27T12:35:00.982896+0000","last_undegraded":"2024-05-27T12:35:00.982896+0000","last_fullsized":"2024-05-27T12:35:00.982896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:04.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980419+0000","last_change":"2024-05-27T12:34:58.906765+0000","last_active":"2024-05-27T12:35:00.980419+0000","last_peered":"2024-05-27T12:35:00.980419+0000","last_clean":"2024-05-27T12:35:00.980419+0000","last_became_active":"2024-05-27T12:34:58.904674+0000","last_became_peered":"2024-05-27T12:34:58.904674+0000","last_unstale":"2024-05-27T12:35:00.980419+0000","last_undegraded":"2024-05-27T12:35:00.980419+0000","last_fullsized":"2024-05-27T12:35:00.980419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943526+0000","last_change":"2024-05-27T12:34:59.943677+0000","last_active":"2024-05-27T12:35:00.943526+0000","last_peered":"2024-05-27T12:35:00.943526+0000","last_clean":"2024-05-27T12:35:00.943526+0000","last_became_active":"2024-05-27T12:34:59.943293+0000","last_became_peered":"2024-05-27T12:34:59.943293+0000","last_unstale":"2024-05-27T12:35:00.943526+0000","last_undegraded":"2024-05-27T12:35:00.943526+0000","last_fullsized":"2024-05-27T12:35:00.943526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980228+0000","last_change":"2024-05-27T12:34:58.916105+0000","last_active":"2024-05-27T12:35:00.980228+0000","last_peered":"2024-05-27T12:35:00.980228+0000","last_clean":"2024-05-27T12:35:00.980228+0000","last_became_active":"2024-05-27T12:34:58.913880+0000","last_became_peered":"2024-05-27T12:34:58.913880+0000","last_unstale":"2024-05-27T12:35:00.980228+0000","last_undegraded":"2024-05-27T12:35:00.980228+0000","last_fullsized":"2024-05-27T12:35:00.980228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946999+0000","last_change":"2024-05-27T12:34:59.946006+0000","last_active":"2024-05-27T12:35:00.946999+0000","last_peered":"2024-05-27T12:35:00.946999+0000","last_clean":"2024-05-27T12:35:00.946999+0000","last_became_active":"2024-05-27T12:34:59.945730+0000","last_became_peered":"2024-05-27T12:34:59.945730+0000","last_unstale":"2024-05-27T12:35:00.946999+0000","last_undegraded":"2024-05-27T12:35:00.946999+0000","last_fullsized":"2024-05-27T12:35:00.946999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:24.764029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943841+0000","last_change":"2024-05-27T12:34:58.906953+0000","last_active":"2024-05-27T12:35:00.943841+0000","last_peered":"2024-05-27T12:35:00.943841+0000","last_clean":"2024-05-27T12:35:00.943841+0000","last_became_active":"2024-05-27T12:34:58.906672+0000","last_became_peered":"2024-05-27T12:34:58.906672+0000","last_unstale":"2024-05-27T12:35:00.943841+0000","last_undegraded":"2024-05-27T12:35:00.943841+0000","last_fullsized":"2024-05-27T12:35:00.943841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980531+0000","last_change":"2024-05-27T12:34:59.947648+0000","last_active":"2024-05-27T12:35:00.980531+0000","last_peered":"2024-05-27T12:35:00.980531+0000","last_clean":"2024-05-27T12:35:00.980531+0000","last_became_active":"2024-05-27T12:34:59.946323+0000","last_became_peered":"2024-05-27T12:34:59.946323+0000","last_unstale":"2024-05-27T12:35:00.980531+0000","last_undegraded":"2024-05-27T12:35:00.980531+0000","last_fullsized":"2024-05-27T12:35:00.980531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983010+0000","last_change":"2024-05-27T12:34:58.922207+0000","last_active":"2024-05-27T12:35:00.983010+0000","last_peered":"2024-05-27T12:35:00.983010+0000","last_clean":"2024-05-27T12:35:00.983010+0000","last_became_active":"2024-05-27T12:34:58.921862+0000","last_became_peered":"2024-05-27T12:34:58.921862+0000","last_unstale":"2024-05-27T12:35:00.983010+0000","last_undegraded":"2024-05-27T12:35:00.983010+0000","last_fullsized":"2024-05-27T12:35:00.983010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:44.514186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943666+0000","last_change":"2024-05-27T12:34:59.945417+0000","last_active":"2024-05-27T12:35:00.943666+0000","last_peered":"2024-05-27T12:35:00.943666+0000","last_clean":"2024-05-27T12:35:00.943666+0000","last_became_active":"2024-05-27T12:34:59.945068+0000","last_became_peered":"2024-05-27T12:34:59.945068+0000","last_unstale":"2024-05-27T12:35:00.943666+0000","last_undegraded":"2024-05-27T12:35:00.943666+0000","last_fullsized":"2024-05-27T12:35:00.943666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.965456+0000","last_change":"2024-05-27T12:34:58.914462+0000","last_active":"2024-05-27T12:35:00.965456+0000","last_peered":"2024-05-27T12:35:00.965456+0000","last_clean":"2024-05-27T12:35:00.965456+0000","last_became_active":"2024-05-27T12:34:58.914188+0000","last_became_peered":"2024-05-27T12:34:58.914188+0000","last_unstale":"2024-05-27T12:35:00.965456+0000","last_undegraded":"2024-05-27T12:35:00.965456+0000","last_fullsized":"2024-05-27T12:35:00.965456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983335+0000","last_change":"2024-05-27T12:34:59.952226+0000","last_active":"2024-05-27T12:35:00.983335+0000","last_peered":"2024-05-27T12:35:00.983335+0000","last_clean":"2024-05-27T12:35:00.983335+0000","last_became_active":"2024-05-27T12:34:59.951936+0000","last_became_peered":"2024-05-27T12:34:59.951936+0000","last_unstale":"2024-05-27T12:35:00.983335+0000","last_undegraded":"2024-05-27T12:35:00.983335+0000","last_fullsized":"2024-05-27T12:35:00.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945936+0000","last_change":"2024-05-27T12:34:58.919780+0000","last_active":"2024-05-27T12:35:00.945936+0000","last_peered":"2024-05-27T12:35:00.945936+0000","last_clean":"2024-05-27T12:35:00.945936+0000","last_became_active":"2024-05-27T12:34:58.919505+0000","last_became_peered":"2024-05-27T12:34:58.919505+0000","last_unstale":"2024-05-27T12:35:00.945936+0000","last_undegraded":"2024-05-27T12:35:00.945936+0000","last_fullsized":"2024-05-27T12:35:00.945936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984200+0000","last_change":"2024-05-27T12:34:59.946900+0000","last_active":"2024-05-27T12:35:00.984200+0000","last_peered":"2024-05-27T12:35:00.984200+0000","last_clean":"2024-05-27T12:35:00.984200+0000","last_became_active":"2024-05-27T12:34:59.945045+0000","last_became_peered":"2024-05-27T12:34:59.945045+0000","last_unstale":"2024-05-27T12:35:00.984200+0000","last_undegraded":"2024-05-27T12:35:00.984200+0000","last_fullsized":"2024-05-27T12:35:00.984200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981186+0000","last_change":"2024-05-27T12:34:58.920871+0000","last_active":"2024-05-27T12:35:00.981186+0000","last_peered":"2024-05-27T12:35:00.981186+0000","last_clean":"2024-05-27T12:35:00.981186+0000","last_became_active":"2024-05-27T12:34:58.917367+0000","last_became_peered":"2024-05-27T12:34:58.917367+0000","last_unstale":"2024-05-27T12:35:00.981186+0000","last_undegraded":"2024-05-27T12:35:00.981186+0000","last_fullsized":"2024-05-27T12:35:00.981186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946167+0000","last_change":"2024-05-27T12:34:59.939621+0000","last_active":"2024-05-27T12:35:00.946167+0000","last_peered":"2024-05-27T12:35:00.946167+0000","last_clean":"2024-05-27T12:35:00.946167+0000","last_became_active":"2024-05-27T12:34:59.939269+0000","last_became_peered":"2024-05-27T12:34:59.939269+0000","last_unstale":"2024-05-27T12:35:00.946167+0000","last_undegraded":"2024-05-27T12:35:00.946167+0000","last_fullsized":"2024-05-27T12:35:00.946167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:45.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944012+0000","last_change":"2024-05-27T12:34:58.922915+0000","last_active":"2024-05-27T12:35:00.944012+0000","last_peered":"2024-05-27T12:35:00.944012+0000","last_clean":"2024-05-27T12:35:00.944012+0000","last_became_active":"2024-05-27T12:34:58.922438+0000","last_became_peered":"2024-05-27T12:34:58.922438+0000","last_unstale":"2024-05-27T12:35:00.944012+0000","last_undegraded":"2024-05-27T12:35:00.944012+0000","last_fullsized":"2024-05-27T12:35:00.944012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980353+0000","last_change":"2024-05-27T12:34:59.947355+0000","last_active":"2024-05-27T12:35:00.980353+0000","last_peered":"2024-05-27T12:35:00.980353+0000","last_clean":"2024-05-27T12:35:00.980353+0000","last_became_active":"2024-05-27T12:34:59.947069+0000","last_became_peered":"2024-05-27T12:34:59.947069+0000","last_unstale":"2024-05-27T12:35:00.980353+0000","last_undegraded":"2024-05-27T12:35:00.980353+0000","last_fullsized":"2024-05-27T12:35:00.980353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943707+0000","last_change":"2024-05-27T12:34:58.911817+0000","last_active":"2024-05-27T12:35:00.943707+0000","last_peered":"2024-05-27T12:35:00.943707+0000","last_clean":"2024-05-27T12:35:00.943707+0000","last_became_active":"2024-05-27T12:34:58.911528+0000","last_became_peered":"2024-05-27T12:34:58.911528+0000","last_unstale":"2024-05-27T12:35:00.943707+0000","last_undegraded":"2024-05-27T12:35:00.943707+0000","last_fullsized":"2024-05-27T12:35:00.943707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981479+0000","last_change":"2024-05-27T12:34:59.944469+0000","last_active":"2024-05-27T12:35:00.981479+0000","last_peered":"2024-05-27T12:35:00.981479+0000","last_clean":"2024-05-27T12:35:00.981479+0000","last_became_active":"2024-05-27T12:34:59.943952+0000","last_became_peered":"2024-05-27T12:34:59.943952+0000","last_unstale":"2024-05-27T12:35:00.981479+0000","last_undegraded":"2024-05-27T12:35:00.981479+0000","last_fullsized":"2024-05-27T12:35:00.981479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982842+0000","last_change":"2024-05-27T12:34:58.928167+0000","last_active":"2024-05-27T12:35:00.982842+0000","last_peered":"2024-05-27T12:35:00.982842+0000","last_clean":"2024-05-27T12:35:00.982842+0000","last_became_active":"2024-05-27T12:34:58.927922+0000","last_became_peered":"2024-05-27T12:34:58.927922+0000","last_unstale":"2024-05-27T12:35:00.982842+0000","last_undegraded":"2024-05-27T12:35:00.982842+0000","last_fullsized":"2024-05-27T12:35:00.982842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:03.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945600+0000","last_change":"2024-05-27T12:34:59.951341+0000","last_active":"2024-05-27T12:35:00.945600+0000","last_peered":"2024-05-27T12:35:00.945600+0000","last_clean":"2024-05-27T12:35:00.945600+0000","last_became_active":"2024-05-27T12:34:59.949217+0000","last_became_peered":"2024-05-27T12:34:59.949217+0000","last_unstale":"2024-05-27T12:35:00.945600+0000","last_undegraded":"2024-05-27T12:35:00.945600+0000","last_fullsized":"2024-05-27T12:35:00.945600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980544+0000","last_change":"2024-05-27T12:34:59.954080+0000","last_active":"2024-05-27T12:35:00.980544+0000","last_peered":"2024-05-27T12:35:00.980544+0000","last_clean":"2024-05-27T12:35:00.980544+0000","last_became_active":"2024-05-27T12:34:59.946377+0000","last_became_peered":"2024-05-27T12:34:59.946377+0000","last_unstale":"2024-05-27T12:35:00.980544+0000","last_undegraded":"2024-05-27T12:35:00.980544+0000","last_fullsized":"2024-05-27T12:35:00.980544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982176+0000","last_change":"2024-05-27T12:34:58.926815+0000","last_active":"2024-05-27T12:35:00.982176+0000","last_peered":"2024-05-27T12:35:00.982176+0000","last_clean":"2024-05-27T12:35:00.982176+0000","last_became_active":"2024-05-27T12:34:58.926419+0000","last_became_peered":"2024-05-27T12:34:58.926419+0000","last_unstale":"2024-05-27T12:35:00.982176+0000","last_undegraded":"2024-05-27T12:35:00.982176+0000","last_fullsized":"2024-05-27T12:35:00.982176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:05.278200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980205+0000","last_change":"2024-05-27T12:34:59.942376+0000","last_active":"2024-05-27T12:35:00.980205+0000","last_peered":"2024-05-27T12:35:00.980205+0000","last_clean":"2024-05-27T12:35:00.980205+0000","last_became_active":"2024-05-27T12:34:59.941083+0000","last_became_peered":"2024-05-27T12:34:59.941083+0000","last_unstale":"2024-05-27T12:35:00.980205+0000","last_undegraded":"2024-05-27T12:35:00.980205+0000","last_fullsized":"2024-05-27T12:35:00.980205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982827+0000","last_change":"2024-05-27T12:34:58.927394+0000","last_active":"2024-05-27T12:35:00.982827+0000","last_peered":"2024-05-27T12:35:00.982827+0000","last_clean":"2024-05-27T12:35:00.982827+0000","last_became_active":"2024-05-27T12:34:58.927124+0000","last_became_peered":"2024-05-27T12:34:58.927124+0000","last_unstale":"2024-05-27T12:35:00.982827+0000","last_undegraded":"2024-05-27T12:35:00.982827+0000","last_fullsized":"2024-05-27T12:35:00.982827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:53.128806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984386+0000","last_change":"2024-05-27T12:34:59.946923+0000","last_active":"2024-05-27T12:35:00.984386+0000","last_peered":"2024-05-27T12:35:00.984386+0000","last_clean":"2024-05-27T12:35:00.984386+0000","last_became_active":"2024-05-27T12:34:59.946554+0000","last_became_peered":"2024-05-27T12:34:59.946554+0000","last_unstale":"2024-05-27T12:35:00.984386+0000","last_undegraded":"2024-05-27T12:35:00.984386+0000","last_fullsized":"2024-05-27T12:35:00.984386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981709+0000","last_change":"2024-05-27T12:34:58.912889+0000","last_active":"2024-05-27T12:35:00.981709+0000","last_peered":"2024-05-27T12:35:00.981709+0000","last_clean":"2024-05-27T12:35:00.981709+0000","last_became_active":"2024-05-27T12:34:58.912617+0000","last_became_peered":"2024-05-27T12:34:58.912617+0000","last_unstale":"2024-05-27T12:35:00.981709+0000","last_undegraded":"2024-05-27T12:35:00.981709+0000","last_fullsized":"2024-05-27T12:35:00.981709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981440+0000","last_change":"2024-05-27T12:34:59.941269+0000","last_active":"2024-05-27T12:35:00.981440+0000","last_peered":"2024-05-27T12:35:00.981440+0000","last_clean":"2024-05-27T12:35:00.981440+0000","last_became_active":"2024-05-27T12:34:59.941004+0000","last_became_peered":"2024-05-27T12:34:59.941004+0000","last_unstale":"2024-05-27T12:35:00.981440+0000","last_undegraded":"2024-05-27T12:35:00.981440+0000","last_fullsized":"2024-05-27T12:35:00.981440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984278+0000","last_change":"2024-05-27T12:34:58.918917+0000","last_active":"2024-05-27T12:35:00.984278+0000","last_peered":"2024-05-27T12:35:00.984278+0000","last_clean":"2024-05-27T12:35:00.984278+0000","last_became_active":"2024-05-27T12:34:58.918499+0000","last_became_peered":"2024-05-27T12:34:58.918499+0000","last_unstale":"2024-05-27T12:35:00.984278+0000","last_undegraded":"2024-05-27T12:35:00.984278+0000","last_fullsized":"2024-05-27T12:35:00.984278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:35:04.838 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-27T12:35:04.839 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T12:35:04.839 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-27T12:35:04.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-27T12:35:04.839 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-27T12:35:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.080+0000 7f5882400640 1 Processor -- start 2024-05-27T12:35:05.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.081+0000 7f5882400640 1 -- start start 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f5882400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f587c15a250 0x7f587c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f5882400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f587c06f280 0x7f587c0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f5882400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587c160360 con 0x7f587c0cf940 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f5882400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f587c160f70 con 0x7f587c15a250 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f5882400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f587c0cfd90 con 0x7f587c06f280 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.082+0000 7f587b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f587c15a250 0x7f587c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f587c15a250 0x7f587c06ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56422/0 (socket says 172.21.15.94:56422) 2024-05-27T12:35:05.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 -- 172.21.15.94:0/992666345 learned_addr learned my addr 172.21.15.94:0/992666345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:05.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f5873e00640 1 --2- 172.21.15.94:0/992666345 >> v2:172.21.15.97:3300/0 conn(0x7f587c06f280 0x7f587c0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 -- 172.21.15.94:0/992666345 >> v2:172.21.15.97:3300/0 conn(0x7f587c06f280 msgr2=0x7f587c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:05.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 --2- 172.21.15.94:0/992666345 >> v2:172.21.15.97:3300/0 conn(0x7f587c06f280 0x7f587c0cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 -- 172.21.15.94:0/992666345 >> v1:172.21.15.97:6789/0 conn(0x7f587c0cf940 legacy=0x7f587c071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:05.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.083+0000 7f587b400640 1 -- 172.21.15.94:0/992666345 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c160640 con 0x7f587c15a250 2024-05-27T12:35:05.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.084+0000 7f587b400640 1 --2- 172.21.15.94:0/992666345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c15a250 0x7f587c06ed40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f587c0b2ad0 tx=0x7f586c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2588ef04ebe5503f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.084+0000 7f587aa00640 1 -- 172.21.15.94:0/992666345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c037070 con 0x7f587c15a250 2024-05-27T12:35:05.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.084+0000 7f587aa00640 1 -- 172.21.15.94:0/992666345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c032440 con 0x7f587c15a250 2024-05-27T12:35:05.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f587aa00640 1 -- 172.21.15.94:0/992666345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c032810 con 0x7f587c15a250 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 -- 172.21.15.94:0/992666345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c15a250 msgr2=0x7f587c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 --2- 172.21.15.94:0/992666345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c15a250 0x7f587c06ed40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f587c0b2ad0 tx=0x7f586c02a230 comp rx=0 tx=0).stop 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 -- 172.21.15.94:0/992666345 shutdown_connections 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 --2- 172.21.15.94:0/992666345 >> v2:172.21.15.97:3300/0 conn(0x7f587c06f280 0x7f587c0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 --2- 172.21.15.94:0/992666345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c15a250 0x7f587c06ed40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 -- 172.21.15.94:0/992666345 >> 172.21.15.94:0/992666345 conn(0x7f587c0b3830 msgr2=0x7f587c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 -- 172.21.15.94:0/992666345 shutdown_connections 2024-05-27T12:35:05.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.085+0000 7f5882400640 1 -- 172.21.15.94:0/992666345 wait complete. 2024-05-27T12:35:05.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.087+0000 7f5882400640 1 Processor -- start 2024-05-27T12:35:05.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.087+0000 7f5882400640 1 -- start start 2024-05-27T12:35:05.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c0cf940 0x7f587c14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f587c15a250 0x7f587c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f587c15a630 con 0x7f587c06f280 2024-05-27T12:35:05.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f587c160c00 con 0x7f587c0cf940 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f5882400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f587c06e7f0 con 0x7f587c15a250 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f587b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f587b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56430/0 (socket says 172.21.15.94:56430) 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.088+0000 7f587b400640 1 -- 172.21.15.94:0/3373561644 learned_addr learned my addr 172.21.15.94:0/3373561644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f5873e00640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c0cf940 0x7f587c14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 -- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f587c15a250 msgr2=0x7f587c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f587c15a250 0x7f587c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 -- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c0cf940 msgr2=0x7f587c14cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c0cf940 0x7f587c14cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 -- 172.21.15.94:0/3373561644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0b5860 con 0x7f587c06f280 2024-05-27T12:35:05.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f587b400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c041020 con 0x7f587c06f280 2024-05-27T12:35:05.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c030970 con 0x7f587c06f280 2024-05-27T12:35:05.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c037040 con 0x7f587c06f280 2024-05-27T12:35:05.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.089+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c14ddb0 con 0x7f587c06f280 2024-05-27T12:35:05.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.090+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c14e370 con 0x7f587c06f280 2024-05-27T12:35:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.091+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c14e6a0 con 0x7f587c06f280 2024-05-27T12:35:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.092+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f586c044020 con 0x7f587c06f280 2024-05-27T12:35:05.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.093+0000 7f5873400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.093+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f586c0ac590 con 0x7f587c06f280 2024-05-27T12:35:05.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.093+0000 7f5873e00640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.094+0000 7f5873e00640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 0x7f585806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.099+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f586c04b030 con 0x7f587c06f280 2024-05-27T12:35:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.350+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f587c075460 con 0x7f587c06f280 2024-05-27T12:35:05.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.359+0000 7f5873400640 1 -- 172.21.15.94:0/3373561644 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174134 (secure 0 0 0) 0x7f586c07a9c0 con 0x7f587c06f280 2024-05-27T12:35:05.361 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 msgr2=0x7f585806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 0x7f585806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864008040 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 msgr2=0x7f587c15ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c0046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 shutdown_connections 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f587c0cf940 0x7f587c14cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f587c15a250 0x7f587c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 --2- 172.21.15.94:0/3373561644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c06f280 0x7f587c15ba70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 >> 172.21.15.94:0/3373561644 conn(0x7f587c0b3830 msgr2=0x7f587c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 shutdown_connections 2024-05-27T12:35:05.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.364+0000 7f5882400640 1 -- 172.21.15.94:0/3373561644 wait complete. 2024-05-27T12:35:05.382 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":191062125},{"type":"v1","addr":"172.21.15.97:6833","nonce":191062125}]},"active_addr":"172.21.15.97:6833/191062125","active_change":"2024-05-27T12:34:44.830837+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3752332787}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2126432984}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1649828588}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1719600017}]}]} 2024-05-27T12:35:05.385 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-27T12:35:05.385 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-27T12:35:05.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.584+0000 7fb983e00640 1 Processor -- start 2024-05-27T12:35:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.584+0000 7fb983e00640 1 -- start start 2024-05-27T12:35:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb9840d4ae0 0x7fb9840d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98415ee10 con 0x7fb9840b4510 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9840d2a60 con 0x7fb98415ea30 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb9840d2830 con 0x7fb9840d4ae0 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb9840b4510 0x7fb9840cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48060/0 (socket says 172.21.15.94:48060) 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb983400640 1 -- 172.21.15.94:0/2831935991 learned_addr learned my addr 172.21.15.94:0/2831935991 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.586+0000 7fb982000640 1 --2- 172.21.15.94:0/2831935991 >> v2:172.21.15.94:3300/0 conn(0x7fb9840d4ae0 0x7fb9840d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.587+0000 7fb981600640 1 -- 172.21.15.94:0/2831935991 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438730656 0 0) 0x7fb98415ee10 con 0x7fb9840b4510 2024-05-27T12:35:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.587+0000 7fb981600640 1 -- 172.21.15.94:0/2831935991 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb964003520 con 0x7fb9840b4510 2024-05-27T12:35:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.587+0000 7fb982000640 1 -- 172.21.15.94:0/2831935991 >> v1:172.21.15.97:6789/0 conn(0x7fb98415ea30 legacy=0x7fb9840d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.587+0000 7fb982000640 1 -- 172.21.15.94:0/2831935991 >> v1:172.21.15.94:6789/0 conn(0x7fb9840b4510 legacy=0x7fb9840cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.587+0000 7fb982000640 1 -- 172.21.15.94:0/2831935991 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9840d2d40 con 0x7fb9840d4ae0 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.588+0000 7fb982000640 1 --2- 172.21.15.94:0/2831935991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840d4ae0 0x7fb9840d22f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009de0 tx=0x7fb96c02a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a527cb011a7e097 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.588+0000 7fb981600640 1 -- 172.21.15.94:0/2831935991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c007740 con 0x7fb9840d4ae0 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.588+0000 7fb981600640 1 -- 172.21.15.94:0/2831935991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c0078e0 con 0x7fb9840d4ae0 2024-05-27T12:35:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.588+0000 7fb981600640 1 -- 172.21.15.94:0/2831935991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c007cb0 con 0x7fb9840d4ae0 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 -- 172.21.15.94:0/2831935991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840d4ae0 msgr2=0x7fb9840d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 --2- 172.21.15.94:0/2831935991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840d4ae0 0x7fb9840d22f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009de0 tx=0x7fb96c02a7e0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 -- 172.21.15.94:0/2831935991 shutdown_connections 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 --2- 172.21.15.94:0/2831935991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840d4ae0 0x7fb9840d22f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 -- 172.21.15.94:0/2831935991 >> 172.21.15.94:0/2831935991 conn(0x7fb98409da70 msgr2=0x7fb98408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 -- 172.21.15.94:0/2831935991 shutdown_connections 2024-05-27T12:35:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.589+0000 7fb983e00640 1 -- 172.21.15.94:0/2831935991 wait complete. 2024-05-27T12:35:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.590+0000 7fb983e00640 1 Processor -- start 2024-05-27T12:35:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.591+0000 7fb983e00640 1 -- start start 2024-05-27T12:35:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9840d4ae0 0x7fb9840b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb98415ea30 0x7fb9840b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840d35e0 con 0x7fb9840b4510 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840d3300 con 0x7fb98415ea30 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb9840cfd20 con 0x7fb9840d4ae0 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56438/0 (socket says 172.21.15.94:56438) 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 -- 172.21.15.94:0/3443720188 learned_addr learned my addr 172.21.15.94:0/3443720188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 -- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9840d4ae0 msgr2=0x7fb9840b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9840d4ae0 0x7fb9840b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 -- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb98415ea30 msgr2=0x7fb9840b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb983400640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb98415ea30 0x7fb9840b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb98415ea30 0x7fb9840b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.592+0000 7fb982a00640 1 -- 172.21.15.94:0/3443720188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb984153fc0 con 0x7fb9840b4510 2024-05-27T12:35:05.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb982a00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c400 tx=0x7fb97400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974017070 con 0x7fb9840b4510 2024-05-27T12:35:05.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740052d0 con 0x7fb9840b4510 2024-05-27T12:35:05.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974003830 con 0x7fb9840b4510 2024-05-27T12:35:05.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98415f210 con 0x7fb9840b4510 2024-05-27T12:35:05.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.593+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98415f7d0 con 0x7fb9840b4510 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.594+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb984075460 con 0x7fb9840b4510 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.595+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb97401a070 con 0x7fb9840b4510 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.596+0000 7fb97b400640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.596+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb97408af50 con 0x7fb9840b4510 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.597+0000 7fb982000640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:05.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.598+0000 7fb982000640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb96c034000 tx=0x7fb96c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:05.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.602+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9740592e0 con 0x7fb9840b4510 2024-05-27T12:35:05.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.801+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9840ba5a0 con 0x7fb9840b4510 2024-05-27T12:35:05.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.803+0000 7fb97b400640 1 -- 172.21.15.94:0/3443720188 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb974058be0 con 0x7fb9840b4510 2024-05-27T12:35:05.803 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:05.803 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:05.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 msgr2=0x7fb96006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb96c034000 tx=0x7fb96c02e000 comp rx=0 tx=0).stop 2024-05-27T12:35:05.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 msgr2=0x7fb984153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:05.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c400 tx=0x7fb97400c8c0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 shutdown_connections 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.805+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb98415ea30 0x7fb9840b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9840d4ae0 0x7fb9840b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 --2- 172.21.15.94:0/3443720188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9840b4510 0x7fb984153a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 >> 172.21.15.94:0/3443720188 conn(0x7fb98409da70 msgr2=0x7fb98408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 shutdown_connections 2024-05-27T12:35:05.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:05.806+0000 7fb983e00640 1 -- 172.21.15.94:0/3443720188 wait complete. 2024-05-27T12:35:05.822 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-27T12:35:05.822 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.015+0000 7f3f4ec00640 1 Processor -- start 2024-05-27T12:35:06.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.015+0000 7f3f4ec00640 1 -- start start 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.016+0000 7f3f4ec00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f48144bb0 0x7f3f480cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f4ec00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f480ce200 con 0x7f3f480cd6c0 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f4ec00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f480cdc40 con 0x7f3f48072630 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f4ec00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3f48144f90 con 0x7f3f48144bb0 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f47400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3f480cd6c0 0x7f3f48071f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48070/0 (socket says 172.21.15.94:48070) 2024-05-27T12:35:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f47400640 1 -- 172.21.15.94:0/2429007649 learned_addr learned my addr 172.21.15.94:0/2429007649 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f47e00640 1 --2- 172.21.15.94:0/2429007649 >> v2:172.21.15.97:3300/0 conn(0x7f3f48144bb0 0x7f3f480cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.017+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117343722 0 0) 0x7f3f480ce200 con 0x7f3f480cd6c0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c003520 con 0x7f3f480cd6c0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2821794667 0 0) 0x7f3f2c003520 con 0x7f3f480cd6c0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 >> v2:172.21.15.97:3300/0 conn(0x7f3f48144bb0 msgr2=0x7f3f480cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 --2- 172.21.15.94:0/2429007649 >> v2:172.21.15.97:3300/0 conn(0x7f3f48144bb0 0x7f3f480cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 >> v1:172.21.15.97:6789/0 conn(0x7f3f48072630 legacy=0x7f3f480cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480cdf20 con 0x7f3f480cd6c0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f3f34001670 con 0x7f3f480cd6c0 2024-05-27T12:35:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 >> v1:172.21.15.94:6789/0 conn(0x7f3f480cd6c0 legacy=0x7f3f48071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 0x7f3f2c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c005210 0x7f3f2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48144f90 con 0x7f3f2c003a90 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c00e350 con 0x7f3f2c005210 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c00e550 con 0x7f3f2c004800 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.018+0000 7f3f47e00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 0x7f3f2c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47400640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47e00640 1 -- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f2c004800 msgr2=0x7f3f2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47e00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47e00640 1 -- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c005210 msgr2=0x7f3f2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47e00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c005210 0x7f3f2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47e00640 1 -- 172.21.15.94:0/2429007649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c003520 con 0x7f3f2c003a90 2024-05-27T12:35:06.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f4cc00640 1 -- 172.21.15.94:0/2429007649 reap_dead start 2024-05-27T12:35:06.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.019+0000 7f3f47400640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f2c004800 0x7f3f2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.020+0000 7f3f47e00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 0x7f3f2c003eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009a90 tx=0x7f3f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32420ee454b7f267 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.020+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38036070 con 0x7f3f2c003a90 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.020+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38031400 con 0x7f3f2c003a90 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.020+0000 7f3f46a00640 1 -- 172.21.15.94:0/2429007649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38009690 con 0x7f3f2c003a90 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 -- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 msgr2=0x7f3f2c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 0x7f3f2c003eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009a90 tx=0x7f3f2c005830 comp rx=0 tx=0).stop 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 -- 172.21.15.94:0/2429007649 shutdown_connections 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/2429007649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f2c003a90 0x7f3f2c003eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 -- 172.21.15.94:0/2429007649 >> 172.21.15.94:0/2429007649 conn(0x7f3f480b3830 msgr2=0x7f3f480b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 -- 172.21.15.94:0/2429007649 shutdown_connections 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.021+0000 7f3f4ec00640 1 -- 172.21.15.94:0/2429007649 wait complete. 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.023+0000 7f3f4ec00640 1 Processor -- start 2024-05-27T12:35:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.023+0000 7f3f4ec00640 1 -- start start 2024-05-27T12:35:06.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c003a90 0x7f3f4815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f481f4810 0x7f3f4813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48145330 con 0x7f3f48140110 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480ceb20 con 0x7f3f2c003a90 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4ec00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480cd180 con 0x7f3f481f4810 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56454/0 (socket says 172.21.15.94:56454) 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 -- 172.21.15.94:0/4086157480 learned_addr learned my addr 172.21.15.94:0/4086157480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f47400640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f481f4810 0x7f3f4813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 -- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f481f4810 msgr2=0x7f3f4813faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f481f4810 0x7f3f4813faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f47e00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c003a90 0x7f3f4815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 -- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c003a90 msgr2=0x7f3f4815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c003a90 0x7f3f4815bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.024+0000 7f3f4cc00640 1 -- 172.21.15.94:0/4086157480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f48142fc0 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f4cc00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c700 tx=0x7f3f4000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f40005d40 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f48159540 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000a820 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.025+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4815c780 con 0x7f3f48140110 2024-05-27T12:35:06.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.026+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f48075460 con 0x7f3f48140110 2024-05-27T12:35:06.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.027+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f4001a070 con 0x7f3f48140110 2024-05-27T12:35:06.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.028+0000 7f3f3fe00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 0x7f3f2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.028+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f4008adf0 con 0x7f3f48140110 2024-05-27T12:35:06.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.029+0000 7f3f47e00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 0x7f3f2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.030+0000 7f3f47e00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 0x7f3f2806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f38033000 tx=0x7f3f38008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.032+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f40059210 con 0x7f3f48140110 2024-05-27T12:35:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.217+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f4815f580 con 0x7f3f48140110 2024-05-27T12:35:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.220+0000 7f3f3fe00640 1 -- 172.21.15.94:0/4086157480 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f3f40058b10 con 0x7f3f48140110 2024-05-27T12:35:06.219 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:06.219 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 msgr2=0x7f3f2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 0x7f3f2806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f38033000 tx=0x7f3f38008030 comp rx=0 tx=0).stop 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 msgr2=0x7f3f48158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c700 tx=0x7f3f4000cbc0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 shutdown_connections 2024-05-27T12:35:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3f2806c920 0x7f3f2806ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f48140110 0x7f3f48158880 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f481f4810 0x7f3f4813faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 --2- 172.21.15.94:0/4086157480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c003a90 0x7f3f4815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 >> 172.21.15.94:0/4086157480 conn(0x7f3f480b3830 msgr2=0x7f3f480b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 shutdown_connections 2024-05-27T12:35:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.222+0000 7f3f4ec00640 1 -- 172.21.15.94:0/4086157480 wait complete. 2024-05-27T12:35:06.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-27T12:35:06.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-27T12:35:06.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-27T12:35:06.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-27T12:35:06.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-27T12:35:06.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-27T12:35:06.243 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-27T12:35:06.243 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-27T12:35:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.522+0000 7f4d31a00640 1 Processor -- start 2024-05-27T12:35:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.523+0000 7f4d31a00640 1 -- start start 2024-05-27T12:35:06.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.523+0000 7f8adc200640 1 Processor -- start 2024-05-27T12:35:06.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.524+0000 7fac5d200640 1 Processor -- start 2024-05-27T12:35:06.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.524+0000 7f8adc200640 1 -- start start 2024-05-27T12:35:06.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8adc200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 0x7f8ad406ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7f4d31a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d2c072d50 0x7f4d2c0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 0x7f4d2c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7f4d31a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2c0b3b20 con 0x7f4d2c06ff80 2024-05-27T12:35:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7f4d31a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4d2c0b3e00 con 0x7f4d2c072d50 2024-05-27T12:35:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7f4d31a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b40e0 con 0x7f4d2c070410 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8adc200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8ad40d06d0 0x7f8ad40b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8adc200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad406e120 con 0x7f8ad4162e60 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8adc200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40ccf50 con 0x7f8ad40d06d0 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8adc200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad406e400 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.526+0000 7f8ada200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8ad40d06d0 0x7f8ad40b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.525+0000 7fac5d200640 1 -- start start 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7fac5d200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fac5815ec60 0x7fac580d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7fac5d200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac580cf440 con 0x7fac580d28a0 2024-05-27T12:35:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7fac5d200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac5806d410 con 0x7fac5815e880 2024-05-27T12:35:06.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7fac5d200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fac5806d1b0 con 0x7fac5815ec60 2024-05-27T12:35:06.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f4d2be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d2c072d50 0x7f4d2c0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f4d2be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d2c072d50 0x7f4d2c0b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56458/0 (socket says 172.21.15.94:56458) 2024-05-27T12:35:06.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f4d2be00640 1 -- 172.21.15.94:0/590765562 learned_addr learned my addr 172.21.15.94:0/590765562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f8ada200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8ad40d06d0 0x7f8ad40b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56468/0 (socket says 172.21.15.94:56468) 2024-05-27T12:35:06.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f8ada200640 1 -- 172.21.15.94:0/2399638078 learned_addr learned my addr 172.21.15.94:0/2399638078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.527+0000 7f8ad8e00640 1 --2- 172.21.15.94:0/2399638078 >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 0x7f8ad406ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.528+0000 7fac56a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fac5815ec60 0x7fac580d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.528+0000 7fac57e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fac580d28a0 0x7fac580cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48080/0 (socket says 172.21.15.94:48080) 2024-05-27T12:35:06.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.528+0000 7f4d31000640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 0x7f4d2c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.528+0000 7fac57e00640 1 -- 172.21.15.94:0/3987453042 learned_addr learned my addr 172.21.15.94:0/3987453042 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d2be00640 1 -- 172.21.15.94:0/590765562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 msgr2=0x7f4d2c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057336709 0 0) 0x7fac5806d410 con 0x7fac5815e880 2024-05-27T12:35:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f8ad3e00640 1 -- 172.21.15.94:0/2399638078 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234525680 0 0) 0x7f8ad406e120 con 0x7f8ad4162e60 2024-05-27T12:35:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f8ada200640 1 -- 172.21.15.94:0/2399638078 >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 msgr2=0x7f8ad406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac3c003520 con 0x7fac5815e880 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956869383 0 0) 0x7fac580cf440 con 0x7fac580d28a0 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac5806d410 con 0x7fac580d28a0 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1421142949 0 0) 0x7fac5806d410 con 0x7fac580d28a0 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 >> v2:172.21.15.97:3300/0 conn(0x7fac5815ec60 msgr2=0x7fac580d4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 --2- 172.21.15.94:0/3987453042 >> v2:172.21.15.97:3300/0 conn(0x7fac5815ec60 0x7fac580d4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 >> v1:172.21.15.97:6789/0 conn(0x7fac5815e880 legacy=0x7fac5806caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5806d6f0 con 0x7fac580d28a0 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fac44001610 con 0x7fac580d28a0 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 >> v1:172.21.15.94:6789/0 conn(0x7fac580d28a0 legacy=0x7fac580cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7fac56000640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004ba0 0x7fac3c004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac56000640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac3c0037a0 0x7fac3c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac56000640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 0x7fac3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac5806d1b0 con 0x7fac3c004160 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c00c040 con 0x7fac3c004ba0 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c00c240 con 0x7fac3c0037a0 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f8ada200640 1 --2- 172.21.15.94:0/2399638078 >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 0x7f8ad406ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f8ada200640 1 -- 172.21.15.94:0/2399638078 >> v1:172.21.15.97:6789/0 conn(0x7f8ad4162e60 legacy=0x7f8ad40cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f8ada200640 1 -- 172.21.15.94:0/2399638078 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40b50f0 con 0x7f8ad40d06d0 2024-05-27T12:35:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7f8ad8e00640 1 --2- 172.21.15.94:0/2399638078 >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 0x7f8ad406ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:35:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d2be00640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 0x7f4d2c15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d2aa00640 1 -- 172.21.15.94:0/590765562 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665807621 0 0) 0x7f4d2c0b3b20 con 0x7f4d2c06ff80 2024-05-27T12:35:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d2be00640 1 -- 172.21.15.94:0/590765562 >> v1:172.21.15.97:6789/0 conn(0x7f4d2c06ff80 legacy=0x7f4d2c15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d2be00640 1 -- 172.21.15.94:0/590765562 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0b43c0 con 0x7f4d2c072d50 2024-05-27T12:35:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.529+0000 7f4d31000640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 0x7f4d2c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.530+0000 7fac57e00640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 0x7fac3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.531+0000 7fac57400640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004ba0 0x7fac3c004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.531+0000 7fac56a00640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac3c0037a0 0x7fac3c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.531+0000 7fac57e00640 1 -- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac3c0037a0 msgr2=0x7fac3c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.531+0000 7fac57e00640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac3c0037a0 0x7fac3c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.531+0000 7fac57e00640 1 -- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004ba0 msgr2=0x7fac3c004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.532+0000 7f8ada200640 1 --2- 172.21.15.94:0/2399638078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad40b4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40b3600 tx=0x7f8acc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31ed4fe6dea3fee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.532+0000 7fac57e00640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004ba0 0x7fac3c004fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f4d2be00640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c072d50 0x7f4d2c0b48e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009870 tx=0x7f4d1c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459d705a3001d605 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.532+0000 7f194f200640 1 Processor -- start 2024-05-27T12:35:06.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f194f200640 1 -- start start 2024-05-27T12:35:06.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.532+0000 7fac57e00640 1 -- 172.21.15.94:0/3987453042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5806d6f0 con 0x7fac3c004160 2024-05-27T12:35:06.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7fac57e00640 1 -- 172.21.15.94:0/3987453042 reap_dead start 2024-05-27T12:35:06.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f8ad3e00640 1 -- 172.21.15.94:0/2399638078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc036070 con 0x7f8ad40d06d0 2024-05-27T12:35:06.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f4d2aa00640 1 -- 172.21.15.94:0/590765562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c037070 con 0x7f4d2c072d50 2024-05-27T12:35:06.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f4d2aa00640 1 -- 172.21.15.94:0/590765562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c02ae50 con 0x7f4d2c072d50 2024-05-27T12:35:06.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f4d2aa00640 1 -- 172.21.15.94:0/590765562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c032860 con 0x7f4d2c072d50 2024-05-27T12:35:06.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.533+0000 7f8ad3e00640 1 -- 172.21.15.94:0/2399638078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc031600 con 0x7f8ad40d06d0 2024-05-27T12:35:06.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f8ad3e00640 1 -- 172.21.15.94:0/2399638078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc036070 con 0x7f8ad40d06d0 2024-05-27T12:35:06.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7fac57400640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004ba0 0x7fac3c004fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f8adc200640 1 -- 172.21.15.94:0/2399638078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 msgr2=0x7f8ad40b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f8adc200640 1 --2- 172.21.15.94:0/2399638078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad40b4770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40b3600 tx=0x7f8acc02a6a0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f4d31a00640 1 -- 172.21.15.94:0/590765562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c072d50 msgr2=0x7f4d2c0b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f4d31a00640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c072d50 0x7f4d2c0b48e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009870 tx=0x7f4d1c02a300 comp rx=0 tx=0).stop 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7fac57e00640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 0x7fac3c00b9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac580cf3d0 tx=0x7fac3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e409b2bee151176 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44007ca0 con 0x7fac3c004160 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.534+0000 7f194f200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1948005ee0 0x7f1948006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194f200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1948053720 con 0x7f1948006860 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194f200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1948053a00 con 0x7f19480ead00 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194f200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1948054330 con 0x7f1948005ee0 2024-05-27T12:35:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f8adc200640 1 -- 172.21.15.94:0/2399638078 shutdown_connections 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f8adc200640 1 --2- 172.21.15.94:0/2399638078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad40b4770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f8adc200640 1 --2- 172.21.15.94:0/2399638078 >> v2:172.21.15.97:3300/0 conn(0x7f8ad40cfbc0 0x7f8ad406ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f8adc200640 1 -- 172.21.15.94:0/2399638078 >> 172.21.15.94:0/2399638078 conn(0x7f8ad406b930 msgr2=0x7f8ad406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44005af0 con 0x7fac3c004160 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7fac56000640 1 -- 172.21.15.94:0/3987453042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44007070 con 0x7fac3c004160 2024-05-27T12:35:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194d400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1948005ee0 0x7f1948006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194d400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1948005ee0 0x7f1948006320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56492/0 (socket says 172.21.15.94:56492) 2024-05-27T12:35:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.535+0000 7f194d400640 1 -- 172.21.15.94:0/243729826 learned_addr learned my addr 172.21.15.94:0/243729826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7fac5d200640 1 -- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 msgr2=0x7fac3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f194d400640 1 -- 172.21.15.94:0/243729826 >> v1:172.21.15.97:6789/0 conn(0x7f19480ead00 legacy=0x7f194804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7fac5d200640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 0x7fac3c00b9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fac580cf3d0 tx=0x7fac3c005830 comp rx=0 tx=0).stop 2024-05-27T12:35:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f194d400640 1 -- 172.21.15.94:0/243729826 >> v1:172.21.15.94:6789/0 conn(0x7f1948006860 legacy=0x7f1948006cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f194d400640 1 -- 172.21.15.94:0/243729826 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1948053ce0 con 0x7f1948005ee0 2024-05-27T12:35:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f4d31a00640 1 -- 172.21.15.94:0/590765562 shutdown_connections 2024-05-27T12:35:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f8adc200640 1 -- 172.21.15.94:0/2399638078 shutdown_connections 2024-05-27T12:35:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f4d31a00640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c070410 0x7f4d2c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f4d31a00640 1 --2- 172.21.15.94:0/590765562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c072d50 0x7f4d2c0b48e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.536+0000 7f4d31a00640 1 -- 172.21.15.94:0/590765562 >> 172.21.15.94:0/590765562 conn(0x7f4d2c06b930 msgr2=0x7f4d2c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.537+0000 7f4d31a00640 1 -- 172.21.15.94:0/590765562 shutdown_connections 2024-05-27T12:35:06.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.537+0000 7f8adc200640 1 -- 172.21.15.94:0/2399638078 wait complete. 2024-05-27T12:35:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7fac5d200640 1 -- 172.21.15.94:0/3987453042 shutdown_connections 2024-05-27T12:35:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7fac5d200640 1 --2- 172.21.15.94:0/3987453042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac3c004160 0x7fac3c00b9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7fac5d200640 1 -- 172.21.15.94:0/3987453042 >> 172.21.15.94:0/3987453042 conn(0x7fac5809da30 msgr2=0x7fac5808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7fac5d200640 1 -- 172.21.15.94:0/3987453042 shutdown_connections 2024-05-27T12:35:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.537+0000 7f4d31a00640 1 -- 172.21.15.94:0/590765562 wait complete. 2024-05-27T12:35:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7f194d400640 1 --2- 172.21.15.94:0/243729826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948006320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f193c009870 tx=0x7f193c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744a7633380444eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.539+0000 7f194ca00640 1 -- 172.21.15.94:0/243729826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c037070 con 0x7f1948005ee0 2024-05-27T12:35:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.538+0000 7fac5d200640 1 -- 172.21.15.94:0/3987453042 wait complete. 2024-05-27T12:35:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.539+0000 7f194ca00640 1 -- 172.21.15.94:0/243729826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c032440 con 0x7f1948005ee0 2024-05-27T12:35:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.539+0000 7f194ca00640 1 -- 172.21.15.94:0/243729826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c032810 con 0x7f1948005ee0 2024-05-27T12:35:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.539+0000 7f8adc200640 1 Processor -- start 2024-05-27T12:35:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.540+0000 7f194f200640 1 -- 172.21.15.94:0/243729826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 msgr2=0x7f1948006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.540+0000 7f194f200640 1 --2- 172.21.15.94:0/243729826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948006320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f193c009870 tx=0x7f193c02a2a0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.540+0000 7f4d31a00640 1 Processor -- start 2024-05-27T12:35:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.540+0000 7f8adc200640 1 -- start start 2024-05-27T12:35:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f4d31a00640 1 -- start start 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f194f200640 1 -- 172.21.15.94:0/243729826 shutdown_connections 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f194f200640 1 --2- 172.21.15.94:0/243729826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948006320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f194f200640 1 -- 172.21.15.94:0/243729826 >> 172.21.15.94:0/243729826 conn(0x7f1948009fb0 msgr2=0x7f194800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f194f200640 1 -- 172.21.15.94:0/243729826 shutdown_connections 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.541+0000 7f194f200640 1 -- 172.21.15.94:0/243729826 wait complete. 2024-05-27T12:35:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7fac5d200640 1 Processor -- start 2024-05-27T12:35:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f75cb600640 1 Processor -- start 2024-05-27T12:35:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 0x7f8ad40b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ad4162e60 0x7f8ad40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad41667b0 con 0x7f8ad40d06d0 2024-05-27T12:35:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad406dda0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8adc200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad4166930 con 0x7f8ad4162e60 2024-05-27T12:35:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7fac5d200640 1 -- start start 2024-05-27T12:35:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f75cb600640 1 -- start start 2024-05-27T12:35:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f5dac200640 1 Processor -- start 2024-05-27T12:35:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f5dac200640 1 -- start start 2024-05-27T12:35:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.542+0000 7f8ada200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ad4162e60 0x7f8ad40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 0x7f8ad40b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c06ff80 0x7f4d2c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f5dac200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f5dac200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da4162f40 con 0x7f5da4162a50 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f5dac200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da4163220 con 0x7f5da40b6a60 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f5dac200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5da40d2f90 con 0x7f5da415e9a0 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2c070410 0x7f4d2c0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 0x7f4d2c0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b35e0 con 0x7f4d2c06ff80 2024-05-27T12:35:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b3460 con 0x7f4d2c070410 2024-05-27T12:35:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f4d31a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b4e20 con 0x7f4d2c072d50 2024-05-27T12:35:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004160 0x7fac580b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac580ce3c0 0x7fac5815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac580ce840 0x7fac5815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f5da9800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac5806caa0 con 0x7fac580ce840 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac5806dcb0 con 0x7fac3c004160 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7fac5d200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fac5806cc20 con 0x7fac580ce3c0 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f75cb600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75c415ec80 0x7f75c40b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f75cb600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f75cb600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c40d20b0 con 0x7f75c40cf170 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f75cb600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f75c40d1580 con 0x7f75c415ec80 2024-05-27T12:35:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f75cb600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f75c4163670 con 0x7f75c41631e0 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f5da9800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57868/0 (socket says 172.21.15.94:57868) 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f5da9800640 1 -- 172.21.15.94:0/4039119506 learned_addr learned my addr 172.21.15.94:0/4039119506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 0x7f8ad40b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59074/0 (socket says 172.21.15.94:59074) 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.543+0000 7f8ad9800640 1 -- 172.21.15.94:0/595787274 learned_addr learned my addr 172.21.15.94:0/595787274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad8e00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad9800640 1 -- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ad4162e60 msgr2=0x7f8ad40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad9800640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ad4162e60 0x7f8ad40b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad9800640 1 -- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 msgr2=0x7f8ad41504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad9800640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ad40d06d0 0x7f8ad41504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f8ad9800640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40b00c0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7fac57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004160 0x7fac580b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7fac57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac580ce840 0x7fac5815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7fac57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004160 0x7fac580b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59084/0 (socket says 172.21.15.94:59084) 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7fac57400640 1 -- 172.21.15.94:0/437862313 learned_addr learned my addr 172.21.15.94:0/437862313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f75c3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f75c3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57894/0 (socket says 172.21.15.94:57894) 2024-05-27T12:35:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f75c3e00640 1 -- 172.21.15.94:0/2821975793 learned_addr learned my addr 172.21.15.94:0/2821975793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f5da3e00640 1 -- 172.21.15.94:0/4039119506 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731913255 0 0) 0x7f5da4163220 con 0x7f5da40b6a60 2024-05-27T12:35:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f5da3e00640 1 -- 172.21.15.94:0/4039119506 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d8c003520 con 0x7f5da40b6a60 2024-05-27T12:35:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7fac56a00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac580ce3c0 0x7fac5815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f5da9800640 1 -- 172.21.15.94:0/4039119506 >> v1:172.21.15.97:6789/0 conn(0x7f5da40b6a60 legacy=0x7f5da40d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.544+0000 7f4d2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2c070410 0x7f4d2c0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f4d31000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 0x7f4d2c0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f4d31000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 0x7f4d2c0a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57882/0 (socket says 172.21.15.94:57882) 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f4d31000640 1 -- 172.21.15.94:0/607969697 learned_addr learned my addr 172.21.15.94:0/607969697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f75c8c00640 1 --2- 172.21.15.94:0/2821975793 >> v2:172.21.15.97:3300/0 conn(0x7f75c415ec80 0x7f75c40b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f5da9800640 1 -- 172.21.15.94:0/4039119506 >> v1:172.21.15.94:6789/0 conn(0x7f5da4162a50 legacy=0x7f5da40b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f5da9800640 1 -- 172.21.15.94:0/4039119506 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da40d3270 con 0x7f5da415e9a0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f4d2be00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c06ff80 0x7f4d2c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.545+0000 7f8ad9800640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 0x7f8ad40b2b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800c390 tx=0x7f8ac800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8017070 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac80052d0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8003810 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f8adc200640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad40b03a0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f8adc200640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad40b0960 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f4d31000640 1 -- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2c070410 msgr2=0x7f4d2c0aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f194f200640 1 Processor -- start 2024-05-27T12:35:06.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f194f200640 1 -- start start 2024-05-27T12:35:06.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f5da9800640 1 --2- 172.21.15.94:0/4039119506 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d980098d0 tx=0x7f5d9802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec9c7138733dd8d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f5da3e00640 1 -- 172.21.15.94:0/4039119506 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037070 con 0x7f5da415e9a0 2024-05-27T12:35:06.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f5da3e00640 1 -- 172.21.15.94:0/4039119506 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98032440 con 0x7f5da415e9a0 2024-05-27T12:35:06.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f5da3e00640 1 -- 172.21.15.94:0/4039119506 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980327f0 con 0x7f5da415e9a0 2024-05-27T12:35:06.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ad4075460 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7fac57e00640 1 -- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac580ce3c0 msgr2=0x7fac5815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7fac57e00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac580ce3c0 0x7fac5815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7fac57e00640 1 -- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004160 msgr2=0x7fac580b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7fac57e00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac3c004160 0x7fac580b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7fac57e00640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac580b78b0 con 0x7fac580ce840 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f75c3e00640 1 -- 172.21.15.94:0/2821975793 >> v2:172.21.15.97:3300/0 conn(0x7f75c415ec80 msgr2=0x7f75c40b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f75c3e00640 1 --2- 172.21.15.94:0/2821975793 >> v2:172.21.15.97:3300/0 conn(0x7f75c415ec80 0x7f75c40b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f75c3e00640 1 -- 172.21.15.94:0/2821975793 >> v1:172.21.15.97:6789/0 conn(0x7f75c40cf170 legacy=0x7f75c40d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f75c3e00640 1 -- 172.21.15.94:0/2821975793 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40d1810 con 0x7f75c41631e0 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948037090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1948006860 0x7f194817d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19480ead00 0x7f19480e9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19480547c0 con 0x7f1948005ee0 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19480501a0 con 0x7f19480ead00 2024-05-27T12:35:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.548+0000 7f194f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1948054940 con 0x7f1948006860 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 -- 172.21.15.94:0/4039119506 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 msgr2=0x7f5da4162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 --2- 172.21.15.94:0/4039119506 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d980098d0 tx=0x7f5d9802a230 comp rx=0 tx=0).stop 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 -- 172.21.15.94:0/4039119506 shutdown_connections 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 --2- 172.21.15.94:0/4039119506 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da4162510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 -- 172.21.15.94:0/4039119506 >> 172.21.15.94:0/4039119506 conn(0x7f5da40d1350 msgr2=0x7f5da40d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7fac57e00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac580ce840 0x7fac5815ae40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac581624b0 tx=0x7fac44001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f194de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948037090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f194de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948037090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56510/0 (socket says 172.21.15.94:56510) 2024-05-27T12:35:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f194de00640 1 -- 172.21.15.94:0/364293667 learned_addr learned my addr 172.21.15.94:0/364293667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f194e800640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19480ead00 0x7f19480e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44040070 con 0x7fac580ce840 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac440047e0 con 0x7fac580ce840 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44040070 con 0x7fac580ce840 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac580b7b90 con 0x7fac580ce840 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.546+0000 7f4d31000640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2c070410 0x7f4d2c0aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f4d31000640 1 -- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c06ff80 msgr2=0x7f4d2c0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f4d31000640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c06ff80 0x7f4d2c0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f4d31000640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0aa180 con 0x7f4d2c072d50 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.547+0000 7f4d2be00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d2c06ff80 0x7f4d2c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 -- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1948006860 msgr2=0x7f194817d520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1948006860 0x7f194817d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 -- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19480ead00 msgr2=0x7f19480e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19480ead00 0x7f19480e9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1948037690 con 0x7f1948005ee0 2024-05-27T12:35:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f4d31000640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 0x7f4d2c0a9490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007ef0 tx=0x7f4d2400d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194e800640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19480ead00 0x7f19480e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f194de00640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948037090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f194000b3f0 tx=0x7f194000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac580b8150 con 0x7fac580ce840 2024-05-27T12:35:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 -- 172.21.15.94:0/4039119506 shutdown_connections 2024-05-27T12:35:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.549+0000 7f5dac200640 1 -- 172.21.15.94:0/4039119506 wait complete. 2024-05-27T12:35:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f387e200640 1 Processor -- start 2024-05-27T12:35:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f75c3e00640 1 --2- 172.21.15.94:0/2821975793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009e50 tx=0x7f75ac02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5c4d6eb851a2a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.550+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2401a070 con 0x7f4d2c072d50 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d24003b90 con 0x7f4d2c072d50 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004030 con 0x7f4d2c072d50 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f5dac200640 1 Processor -- start 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f75c3400640 1 -- 172.21.15.94:0/2821975793 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac036070 con 0x7f75c41631e0 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f75c3400640 1 -- 172.21.15.94:0/2821975793 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac031600 con 0x7f75c41631e0 2024-05-27T12:35:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.551+0000 7f75c3400640 1 -- 172.21.15.94:0/2821975793 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0319d0 con 0x7f75c41631e0 2024-05-27T12:35:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940015020 con 0x7f1948005ee0 2024-05-27T12:35:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940011830 con 0x7f1948005ee0 2024-05-27T12:35:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940011c20 con 0x7f1948005ee0 2024-05-27T12:35:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f75cb600640 1 -- 172.21.15.94:0/2821975793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 msgr2=0x7f75c40d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f75cb600640 1 --2- 172.21.15.94:0/2821975793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009e50 tx=0x7f75ac02a680 comp rx=0 tx=0).stop 2024-05-27T12:35:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.553+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c0aa460 con 0x7f4d2c072d50 2024-05-27T12:35:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.553+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c0aaa20 con 0x7f4d2c072d50 2024-05-27T12:35:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19480e7440 con 0x7f1948005ee0 2024-05-27T12:35:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19480e7a00 con 0x7f1948005ee0 2024-05-27T12:35:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f5dac200640 1 -- start start 2024-05-27T12:35:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.553+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ac801a070 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac58160b10 con 0x7fac580ce840 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da40b6a60 0x7f5da40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 0x7f5da40b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5da40cf440 con 0x7f5da4162a50 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5da40cf2c0 con 0x7f5da40b6a60 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5dac200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5da40b7bb0 con 0x7f5da415e9a0 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.552+0000 7f387e200640 1 -- start start 2024-05-27T12:35:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5daa200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 0x7f5da40b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.553+0000 7f7cc8800640 1 Processor -- start 2024-05-27T12:35:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f7cc8800640 1 -- start start 2024-05-27T12:35:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5daa200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 0x7f5da40b96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56526/0 (socket says 172.21.15.94:56526) 2024-05-27T12:35:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f5daa200640 1 -- 172.21.15.94:0/818821809 learned_addr learned my addr 172.21.15.94:0/818821809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3878162bb0 0x7f38781673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162f90 0x7f387806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 0x7f387806be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3878166f50 con 0x7f3878162bb0 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f38780b3540 con 0x7f38780afc50 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f387e200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f38780b3820 con 0x7f3878162f90 2024-05-27T12:35:06.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.554+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d2c075460 con 0x7f4d2c072d50 2024-05-27T12:35:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5da8e00640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f8ad1600640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ab406c9c0 0x7f8ab406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ac808b6d0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f3877400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3878162bb0 0x7f38781673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f3877400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3878162bb0 0x7f38781673e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56536/0 (socket says 172.21.15.94:56536) 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 -- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 msgr2=0x7f5da415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5da415e9a0 0x7f5da415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 -- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da40b6a60 msgr2=0x7f5da40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5da9800640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da40b6a60 0x7f5da40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da40b6a60 0x7f5da40c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f3877400640 1 -- 172.21.15.94:0/1002890694 learned_addr learned my addr 172.21.15.94:0/1002890694 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da4160430 con 0x7f5da4162a50 2024-05-27T12:35:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f3876a00640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162f90 0x7f387806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f8ad1600640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] conn(0x7f8ab4074be0 0x7f8ab4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ab40776b0 con 0x7f8ab4074be0 2024-05-27T12:35:06.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f5daa200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 0x7f5da40b96e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c700 tx=0x7f5d9c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8ac808bac0 con 0x7f8ad40cfbc0 2024-05-27T12:35:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.555+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19480052c0 con 0x7f1948005ee0 2024-05-27T12:35:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f3877e00640 1 --2- 172.21.15.94:0/1002890694 >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 0x7f387806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f8ada200640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] conn(0x7f8ab4074be0 0x7f8ab4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f8ad8e00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ab406c9c0 0x7f8ab406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003b70 con 0x7f5da4162a50 2024-05-27T12:35:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 -- 172.21.15.94:0/2821975793 shutdown_connections 2024-05-27T12:35:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 --2- 172.21.15.94:0/2821975793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c41631e0 0x7f75c40d1040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 --2- 172.21.15.94:0/2821975793 >> v2:172.21.15.97:3300/0 conn(0x7f75c415ec80 0x7f75c40b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 -- 172.21.15.94:0/2821975793 >> 172.21.15.94:0/2821975793 conn(0x7f75c406cc20 msgr2=0x7f75c406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 -- 172.21.15.94:0/2821975793 shutdown_connections 2024-05-27T12:35:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c003d30 con 0x7f5da4162a50 2024-05-27T12:35:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00aa90 con 0x7f5da4162a50 2024-05-27T12:35:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f3877400640 1 -- 172.21.15.94:0/1002890694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162f90 msgr2=0x7f387806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f3877400640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162f90 0x7f387806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f8ada200640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] conn(0x7f8ab4074be0 0x7f8ab4077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.557+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da4160710 con 0x7f5da4162a50 2024-05-27T12:35:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.557+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da4160cd0 con 0x7f5da4162a50 2024-05-27T12:35:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.557+0000 7f8ad8e00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ab406c9c0 0x7f8ab406ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009560 tx=0x7f8ac0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.556+0000 7f75cb600640 1 -- 172.21.15.94:0/2821975793 wait complete. 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f3877400640 1 -- 172.21.15.94:0/1002890694 >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 msgr2=0x7f387806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f3877400640 1 --2- 172.21.15.94:0/1002890694 >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 0x7f387806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f3877400640 1 -- 172.21.15.94:0/1002890694 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b3b00 con 0x7f3878162bb0 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f3877e00640 1 --2- 172.21.15.94:0/1002890694 >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 0x7f387806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== osd.0 v2:172.21.15.94:6800/2544283639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ab40776b0 con 0x7f8ab4074be0 2024-05-27T12:35:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5da40684c0 con 0x7f5da4162a50 2024-05-27T12:35:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f75cb600640 1 Processor -- start 2024-05-27T12:35:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7f75cb600640 1 -- start start 2024-05-27T12:35:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2401d070 con 0x7f4d2c072d50 2024-05-27T12:35:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c40cf170 0x7f75c41f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c415ec80 0x7f75c41f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.557+0000 7f7cc8800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc015d150 0x7f7cc0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f7cc8800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7cc006e550 0x7f7cc0072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f7cc8800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc0163330 con 0x7f7cc00b3540 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f7cc8800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc0163610 con 0x7f7cc006e550 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.558+0000 7f7cc8800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc01638f0 con 0x7f7cc015d150 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7f7cc6800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7cc006e550 0x7f7cc0072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7f7cc6800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7cc006e550 0x7f7cc0072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56546/0 (socket says 172.21.15.94:56546) 2024-05-27T12:35:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7f7cc6800640 1 -- 172.21.15.94:0/407587758 learned_addr learned my addr 172.21.15.94:0/407587758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 0x7f75c40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75c415f060 con 0x7f75c41631e0 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40d1dd0 con 0x7f75c415ec80 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75cb600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f75c40cf600 con 0x7f75c40cf170 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75c9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 0x7f75c40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75c9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 0x7f75c40cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56554/0 (socket says 172.21.15.94:56554) 2024-05-27T12:35:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75c9600640 1 -- 172.21.15.94:0/3785406928 learned_addr learned my addr 172.21.15.94:0/3785406928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f75c8c00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c40cf170 0x7f75c41f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d9c00ac30 con 0x7f5da4162a50 2024-05-27T12:35:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.560+0000 7f4d23e00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d2408c8f0 con 0x7f4d2c072d50 2024-05-27T12:35:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f7cc5e00640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc015d150 0x7f7cc0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f4d23e00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] conn(0x7f4d0c074a90 0x7f4d0c076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f75c3e00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c415ec80 0x7f75c41f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d0c077560 con 0x7f4d0c074a90 2024-05-27T12:35:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4d2408cd80 con 0x7f4d2c072d50 2024-05-27T12:35:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.559+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac4403a070 con 0x7fac580ce840 2024-05-27T12:35:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7fac4f400640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fac3806caf0 0x7fac3806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f4d2be00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f75c9600640 1 -- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c40cf170 msgr2=0x7f75c41f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c9600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f75c40cf170 0x7f75c41f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c9600640 1 -- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c415ec80 msgr2=0x7f75c41f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c9600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75c415ec80 0x7f75c41f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c9600640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40b3c30 con 0x7f75c41631e0 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac440762f0 con 0x7fac580ce840 2024-05-27T12:35:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7fac57400640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fac3806caf0 0x7fac3806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c9600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 0x7f75c40cd6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c3f0 tx=0x7f75b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f5da1600640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5d8406c9c0 0x7f5d8406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7fac4f400640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] conn(0x7fac380758a0 0x7fac38077d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac38078410 con 0x7fac380758a0 2024-05-27T12:35:06.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d9c08c420 con 0x7f5da4162a50 2024-05-27T12:35:06.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b40052d0 con 0x7f75c41631e0 2024-05-27T12:35:06.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b4005490 con 0x7f75c41631e0 2024-05-27T12:35:06.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c40b3f10 con 0x7f75c41631e0 2024-05-27T12:35:06.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f5da1600640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] conn(0x7f5d84074c40 0x7f5d84077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d9c004ec0 con 0x7f5d84074c40 2024-05-27T12:35:06.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5d9c08c810 con 0x7f5da4162a50 2024-05-27T12:35:06.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f3877400640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878162bb0 0x7f38781673e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3878072ef0 tx=0x7f386802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a2b5b3fa045b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.562+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4007260 con 0x7f75c41631e0 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c40b44d0 con 0x7f75c41631e0 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f7cc6800640 1 -- 172.21.15.94:0/407587758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc015d150 msgr2=0x7f7cc0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f7cc6800640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc015d150 0x7f7cc0071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f7cc6800640 1 -- 172.21.15.94:0/407587758 >> v1:172.21.15.94:6789/0 conn(0x7f7cc00b3540 legacy=0x7f7cc006de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.561+0000 7f7cc6800640 1 -- 172.21.15.94:0/407587758 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0163bd0 con 0x7f7cc006e550 2024-05-27T12:35:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f5da9800640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5d8406c9c0 0x7f5d8406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f4d2be00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c034000 tx=0x7f4d1c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.563+0000 7f5da8e00640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] conn(0x7f5d84074c40 0x7f5d84077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7f4d2b400640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] conn(0x7f4d0c074a90 0x7f4d0c076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7f4d2b400640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] conn(0x7f4d0c074a90 0x7f4d0c076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7f7cc6800640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc006e550 0x7f7cc0072380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009e50 tx=0x7f7cbc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfcdd81bc756231 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7fac56a00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] conn(0x7fac380758a0 0x7fac38077d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7fac57400640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fac3806caf0 0x7fac3806efb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac580c7fd0 tx=0x7fac48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75c4075460 con 0x7f75c41631e0 2024-05-27T12:35:06.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f5da9800640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5d8406c9c0 0x7f5d8406ee80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.564+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1940018020 con 0x7f1948005ee0 2024-05-27T12:35:06.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fac44074c80 con 0x7fac580ce840 2024-05-27T12:35:06.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f5da8e00640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] conn(0x7f5d84074c40 0x7f5d84077060 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f3876000640 1 -- 172.21.15.94:0/1002890694 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868037070 con 0x7f3878162bb0 2024-05-27T12:35:06.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f3876000640 1 -- 172.21.15.94:0/1002890694 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3868032440 con 0x7f3878162bb0 2024-05-27T12:35:06.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f3876000640 1 -- 172.21.15.94:0/1002890694 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868032810 con 0x7f3878162bb0 2024-05-27T12:35:06.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f1946000640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f193006c9c0 0x7f193006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f194008bb60 con 0x7f1948005ee0 2024-05-27T12:35:06.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f387e200640 1 -- 172.21.15.94:0/1002890694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878162bb0 msgr2=0x7f38781673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== osd.2 v2:172.21.15.94:6819/4017177673 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d0c077560 con 0x7f4d0c074a90 2024-05-27T12:35:06.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f1946000640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] conn(0x7f1930074be0 0x7f1930077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f387e200640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878162bb0 0x7f38781673e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3878072ef0 tx=0x7f386802a1e0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19300776b0 con 0x7f1930074be0 2024-05-27T12:35:06.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f19400902a0 con 0x7f1948005ee0 2024-05-27T12:35:06.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f194d400640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f193006c9c0 0x7f193006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7fac56a00640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] conn(0x7fac380758a0 0x7fac38077d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f7cc4a00640 1 -- 172.21.15.94:0/407587758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc036070 con 0x7f7cc006e550 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f387e200640 1 -- 172.21.15.94:0/1002890694 shutdown_connections 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f7cc4a00640 1 -- 172.21.15.94:0/407587758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc031600 con 0x7f7cc006e550 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.565+0000 7f7cc4a00640 1 -- 172.21.15.94:0/407587758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0319d0 con 0x7f7cc006e550 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f7cc8800640 1 -- 172.21.15.94:0/407587758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc006e550 msgr2=0x7f7cc0072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f7cc8800640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc006e550 0x7f7cc0072380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009e50 tx=0x7f7cbc02a630 comp rx=0 tx=0).stop 2024-05-27T12:35:06.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f387e200640 1 --2- 172.21.15.94:0/1002890694 >> v2:172.21.15.97:3300/0 conn(0x7f38780afc50 0x7f387806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f387e200640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162f90 0x7f387806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f7cc8800640 1 -- 172.21.15.94:0/407587758 shutdown_connections 2024-05-27T12:35:06.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f7cc8800640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc006e550 0x7f7cc0072380 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f7cc8800640 1 --2- 172.21.15.94:0/407587758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc015d150 0x7f7cc0071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f7cc8800640 1 -- 172.21.15.94:0/407587758 >> 172.21.15.94:0/407587758 conn(0x7f7cc00b2660 msgr2=0x7f7cc00b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f194e800640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] conn(0x7f1930074be0 0x7f1930077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f387e200640 1 --2- 172.21.15.94:0/1002890694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878162bb0 0x7f38781673e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.567+0000 7f387e200640 1 -- 172.21.15.94:0/1002890694 >> 172.21.15.94:0/1002890694 conn(0x7f3878067d70 msgr2=0x7f38780b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.566+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75b4007400 con 0x7f75c41631e0 2024-05-27T12:35:06.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f75c0c00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f75a006c8f0 0x7f75a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== osd.5 v2:172.21.15.97:6808/596380992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d9c004ec0 con 0x7f5d84074c40 2024-05-27T12:35:06.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f194d400640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f193006c9c0 0x7f193006ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f193c034000 tx=0x7f193c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f7cc8800640 1 -- 172.21.15.94:0/407587758 shutdown_connections 2024-05-27T12:35:06.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f194e800640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] conn(0x7f1930074be0 0x7f1930077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f7cc8800640 1 -- 172.21.15.94:0/407587758 wait complete. 2024-05-27T12:35:06.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75b4056e10 con 0x7f75c41631e0 2024-05-27T12:35:06.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f75c0c00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] conn(0x7f75a0075690 0x7f75a0077b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.568+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75a0078200 con 0x7f75a0075690 2024-05-27T12:35:06.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.569+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75b40557a0 con 0x7f75c41631e0 2024-05-27T12:35:06.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.569+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== osd.3 v2:172.21.15.94:6816/1503691431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19300776b0 con 0x7f1930074be0 2024-05-27T12:35:06.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.570+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== osd.1 v2:172.21.15.94:6808/1208208615 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac38078410 con 0x7fac380758a0 2024-05-27T12:35:06.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.569+0000 7f75c8c00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f75a006c8f0 0x7f75a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.570+0000 7f75c3e00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] conn(0x7f75a0075690 0x7f75a0077b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.571+0000 7f75c8c00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f75a006c8f0 0x7f75a006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009ad0 tx=0x7f75bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.571+0000 7f75c3e00640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] conn(0x7f75a0075690 0x7f75a0077b30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.571+0000 7f387e200640 1 -- 172.21.15.94:0/1002890694 shutdown_connections 2024-05-27T12:35:06.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.571+0000 7f7cc8800640 1 Processor -- start 2024-05-27T12:35:06.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.572+0000 7f7cc8800640 1 -- start start 2024-05-27T12:35:06.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== osd.4 v2:172.21.15.97:6816/1902436281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75a0078200 con 0x7f75a0075690 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cc006e550 0x7f7cc01601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc00b3540 0x7f7cc0166a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc015d150 0x7f7cc0166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc006ece0 con 0x7f7cc015d150 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc006eb60 con 0x7f7cc006e550 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.573+0000 7f7cc8800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc006ee60 con 0x7f7cc00b3540 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.574+0000 7f7cc5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cc006e550 0x7f7cc01601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.574+0000 7f7cc5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cc006e550 0x7f7cc01601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59154/0 (socket says 172.21.15.94:59154) 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.574+0000 7f7cc5e00640 1 -- 172.21.15.94:0/823499952 learned_addr learned my addr 172.21.15.94:0/823499952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.574+0000 7f7cc6800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc015d150 0x7f7cc0166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.575+0000 7f7cc6800640 1 -- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc00b3540 msgr2=0x7f7cc0166a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.575+0000 7f7cc6800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7cc00b3540 0x7f7cc0166a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.575+0000 7f7cc6800640 1 -- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cc006e550 msgr2=0x7f7cc01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.575+0000 7f7cc6800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cc006e550 0x7f7cc01601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.575+0000 7f7cc6800640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0159070 con 0x7f7cc015d150 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.571+0000 7f387e200640 1 -- 172.21.15.94:0/1002890694 wait complete. 2024-05-27T12:35:06.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.574+0000 7f387e200640 1 Processor -- start 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.576+0000 7f7cc6800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc015d150 0x7f7cc0166fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc030000 tx=0x7f7cbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.577+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc041020 con 0x7f7cc015d150 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.577+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc008a30 con 0x7f7cc015d150 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.577+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc036040 con 0x7f7cc015d150 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.576+0000 7f387e200640 1 -- start start 2024-05-27T12:35:06.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 0x7f387815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878164de0 0x7f3878147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38781591c0 0x7f3878159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38780cfb00 con 0x7f3878164de0 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3878166be0 con 0x7f38781591c0 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f387e200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f38780cfc80 con 0x7f3878162bb0 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc0159310 con 0x7f7cc015d150 2024-05-27T12:35:06.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.578+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01598d0 con 0x7f7cc015d150 2024-05-27T12:35:06.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 0x7f387815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 0x7f387815c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57964/0 (socket says 172.21.15.94:57964) 2024-05-27T12:35:06.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 -- 172.21.15.94:0/1405768430 learned_addr learned my addr 172.21.15.94:0/1405768430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877e00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38781591c0 0x7f3878159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3876a00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878164de0 0x7f3878147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 -- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38781591c0 msgr2=0x7f3878159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38781591c0 0x7f3878159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 -- 172.21.15.94:0/1405768430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878164de0 msgr2=0x7f3878147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3878164de0 0x7f3878147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.579+0000 7f3877400640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3878147880 con 0x7f3878162bb0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.580+0000 7f3877400640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 0x7f387815c0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f38680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.581+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868037040 con 0x7f3878162bb0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.581+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3868042070 con 0x7f3878162bb0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.581+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868030940 con 0x7f3878162bb0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.583+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3878165260 con 0x7f3878162bb0 2024-05-27T12:35:06.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.583+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3878165810 con 0x7f3878162bb0 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.582+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cbc044020 con 0x7f7cc015d150 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.583+0000 7f7caf400640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cc0075460 con 0x7f7cc015d150 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.584+0000 7f7cba000640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7ca406c940 0x7f7ca406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.584+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7cbc0ac7e0 con 0x7f7cc015d150 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.584+0000 7f7cba000640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] conn(0x7f7ca4074b60 0x7f7ca4076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.584+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cbc031600 con 0x7f7ca4074b60 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.584+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7cbc0acbc0 con 0x7f7cc015d150 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.585+0000 7f7cc5400640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] conn(0x7f7ca4074b60 0x7f7ca4076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.585+0000 7f7cc5e00640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7ca406c940 0x7f7ca406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.586+0000 7f7cc5400640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] conn(0x7f7ca4074b60 0x7f7ca4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.586+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3878075460 con 0x7f3878162bb0 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.587+0000 7f7cc5e00640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7ca406c940 0x7f7ca406ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4007a60 tx=0x7f7cb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.589+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== osd.6 v2:172.21.15.97:6817/3234217179 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cbc031600 con 0x7f7ca4074b60 2024-05-27T12:35:06.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.589+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3868030ae0 con 0x7f3878162bb0 2024-05-27T12:35:06.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.590+0000 7f386f400640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f385806c9c0 0x7f385806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.590+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38680ac520 con 0x7f3878162bb0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.591+0000 7f386f400640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] conn(0x7f3858074be0 0x7f3858077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.591+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f386803eec0 con 0x7f3858074be0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.591+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f38680ac910 con 0x7f3878162bb0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.591+0000 7f3876a00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f385806c9c0 0x7f385806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.592+0000 7f3877e00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] conn(0x7f3858074be0 0x7f3858077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.593+0000 7f3877e00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] conn(0x7f3858074be0 0x7f3858077000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.593+0000 7f3876a00640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f385806c9c0 0x7f385806ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38781fc040 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.594+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 --> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1948047a80 con 0x7f1930074be0 2024-05-27T12:35:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.595+0000 7f1946000640 1 -- 172.21.15.94:0/364293667 <== osd.3 v2:172.21.15.94:6816/1503691431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1948047a80 con 0x7f1930074be0 2024-05-27T12:35:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.595+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 --> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d2c0cd870 con 0x7f4d0c074a90 2024-05-27T12:35:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.595+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 --> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ad40b3630 con 0x7f8ab4074be0 2024-05-27T12:35:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.595+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== osd.7 v2:172.21.15.97:6800/3824367189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f386803eec0 con 0x7f3858074be0 2024-05-27T12:35:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.595+0000 7f4d23e00640 1 -- 172.21.15.94:0/607969697 <== osd.2 v2:172.21.15.94:6819/4017177673 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4d2c0cd870 con 0x7f4d0c074a90 2024-05-27T12:35:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f8ad1600640 1 -- 172.21.15.94:0/595787274 <== osd.0 v2:172.21.15.94:6800/2544283639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ad40b3630 con 0x7f8ab4074be0 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] conn(0x7f1930074be0 msgr2=0x7f1930077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:6816/1503691431,v1:172.21.15.94:6817/1503691431] conn(0x7f1930074be0 0x7f1930077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f193006c9c0 msgr2=0x7f193006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f193006c9c0 0x7f193006ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f193c034000 tx=0x7f193c02e000 comp rx=0 tx=0).stop 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] conn(0x7f8ab4074be0 msgr2=0x7f8ab4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ac6a00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.94:6800/2544283639,v1:172.21.15.94:6801/2544283639] conn(0x7f8ab4074be0 0x7f8ab4077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] conn(0x7f4d0c074a90 msgr2=0x7f4d0c076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.94:6819/4017177673,v1:172.21.15.94:6821/4017177673] conn(0x7f4d0c074a90 0x7f4d0c076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 msgr2=0x7f1948037090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194f200640 1 --2- 172.21.15.94:0/364293667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1948005ee0 0x7f1948037090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f194000b3f0 tx=0x7f194000b8b0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.596+0000 7f194e800640 1 -- 172.21.15.94:0/364293667 reap_dead start 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 shutdown_connections 2024-05-27T12:35:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 >> 172.21.15.94:0/364293667 conn(0x7f1948009fb0 msgr2=0x7f1948049e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d0c06c8f0 msgr2=0x7f4d0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c034000 tx=0x7f4d1c02e000 comp rx=0 tx=0).stop 2024-05-27T12:35:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 msgr2=0x7f4d2c0a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31a00640 1 --2- 172.21.15.94:0/607969697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d2c072d50 0x7f4d2c0a9490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007ef0 tx=0x7f4d2400d4a0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f4d31000640 1 -- 172.21.15.94:0/607969697 reap_dead start 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ab406c9c0 msgr2=0x7f8ab406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ac6a00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ab406c9c0 0x7f8ab406ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009560 tx=0x7f8ac0008040 comp rx=0 tx=0).stop 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 msgr2=0x7f8ad40b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ac6a00640 1 --2- 172.21.15.94:0/595787274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40cfbc0 0x7f8ad40b2b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800c390 tx=0x7f8ac800c850 comp rx=0 tx=0).stop 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.597+0000 7f8ada200640 1 -- 172.21.15.94:0/595787274 reap_dead start 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 shutdown_connections 2024-05-27T12:35:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 >> 172.21.15.94:0/595787274 conn(0x7f8ad406b930 msgr2=0x7f8ad41653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 shutdown_connections 2024-05-27T12:35:06.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 >> 172.21.15.94:0/607969697 conn(0x7f4d2c06b930 msgr2=0x7f4d2c06f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 shutdown_connections 2024-05-27T12:35:06.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 shutdown_connections 2024-05-27T12:35:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f8ac6a00640 1 -- 172.21.15.94:0/595787274 wait complete. 2024-05-27T12:35:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.598+0000 7f4d31a00640 1 -- 172.21.15.94:0/607969697 wait complete. 2024-05-27T12:35:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.599+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 shutdown_connections 2024-05-27T12:35:06.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.599+0000 7f194f200640 1 -- 172.21.15.94:0/364293667 wait complete. 2024-05-27T12:35:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.604+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 --> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75c40d2570 con 0x7f75a0075690 2024-05-27T12:35:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.606+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 --> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5da4075460 con 0x7f5d84074c40 2024-05-27T12:35:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.606+0000 7f75c0c00640 1 -- 172.21.15.94:0/3785406928 <== osd.4 v2:172.21.15.97:6816/1902436281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75c40d2570 con 0x7f75a0075690 2024-05-27T12:35:06.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.606+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 --> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac58075460 con 0x7fac380758a0 2024-05-27T12:35:06.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.607+0000 7f5da1600640 1 -- 172.21.15.94:0/818821809 <== osd.5 v2:172.21.15.97:6808/596380992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5da4075460 con 0x7f5d84074c40 2024-05-27T12:35:06.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.607+0000 7fac4f400640 1 -- 172.21.15.94:0/437862313 <== osd.1 v2:172.21.15.94:6808/1208208615 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fac58075460 con 0x7fac380758a0 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.608+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] conn(0x7fac380758a0 msgr2=0x7fac38077d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.608+0000 7fac5d200640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:6808/1208208615,v1:172.21.15.94:6809/1208208615] conn(0x7fac380758a0 0x7fac38077d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] conn(0x7f5d84074c40 msgr2=0x7f5d84077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6808/596380992,v1:172.21.15.97:6809/596380992] conn(0x7f5d84074c40 0x7f5d84077060 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] conn(0x7f75a0075690 msgr2=0x7f75a0077b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6816/1902436281,v1:172.21.15.97:6819/1902436281] conn(0x7f75a0075690 0x7f75a0077b30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fac3806caf0 msgr2=0x7fac3806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fac3806caf0 0x7fac3806efb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac580c7fd0 tx=0x7fac48009040 comp rx=0 tx=0).stop 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac580ce840 msgr2=0x7fac5815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 --2- 172.21.15.94:0/437862313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac580ce840 0x7fac5815ae40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac581624b0 tx=0x7fac44001910 comp rx=0 tx=0).stop 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5d8406c9c0 msgr2=0x7f5d8406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5d8406c9c0 0x7f5d8406ee80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).stop 2024-05-27T12:35:06.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 msgr2=0x7f5da40b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 --2- 172.21.15.94:0/818821809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5da4162a50 0x7f5da40b96e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c700 tx=0x7f5d9c00cbc0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f75a006c8f0 msgr2=0x7f75a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f75a006c8f0 0x7f75a006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009ad0 tx=0x7f75bc008040 comp rx=0 tx=0).stop 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 msgr2=0x7f75c40cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f75cb600640 1 --2- 172.21.15.94:0/3785406928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75c41631e0 0x7f75c40cd6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c3f0 tx=0x7f75b400c8b0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5daa200640 1 -- 172.21.15.94:0/818821809 reap_dead start 2024-05-27T12:35:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac57e00640 1 -- 172.21.15.94:0/437862313 reap_dead start 2024-05-27T12:35:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 shutdown_connections 2024-05-27T12:35:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 >> 172.21.15.94:0/437862313 conn(0x7fac5809da30 msgr2=0x7fac580d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 shutdown_connections 2024-05-27T12:35:06.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.609+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 >> 172.21.15.94:0/818821809 conn(0x7f5da40d1350 msgr2=0x7f5da40b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.610+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 shutdown_connections 2024-05-27T12:35:06.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.610+0000 7f75c9600640 1 -- 172.21.15.94:0/3785406928 reap_dead start 2024-05-27T12:35:06.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.610+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 shutdown_connections 2024-05-27T12:35:06.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.610+0000 7fac5d200640 1 -- 172.21.15.94:0/437862313 wait complete. 2024-05-27T12:35:06.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.610+0000 7f5dac200640 1 -- 172.21.15.94:0/818821809 wait complete. 2024-05-27T12:35:06.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.611+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 shutdown_connections 2024-05-27T12:35:06.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.611+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 >> 172.21.15.94:0/3785406928 conn(0x7f75c406cc20 msgr2=0x7f75c40cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.611+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 shutdown_connections 2024-05-27T12:35:06.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.611+0000 7f75cb600640 1 -- 172.21.15.94:0/3785406928 wait complete. 2024-05-27T12:35:06.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.617+0000 7f7caf400640 1 -- 172.21.15.94:0/823499952 --> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7cc0072ec0 con 0x7f7ca4074b60 2024-05-27T12:35:06.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.617+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 --> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f387806e040 con 0x7f3858074be0 2024-05-27T12:35:06.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.618+0000 7f7cba000640 1 -- 172.21.15.94:0/823499952 <== osd.6 v2:172.21.15.97:6817/3234217179 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7cc0072ec0 con 0x7f7ca4074b60 2024-05-27T12:35:06.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.618+0000 7f386f400640 1 -- 172.21.15.94:0/1405768430 <== osd.7 v2:172.21.15.97:6800/3824367189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f387806e040 con 0x7f3858074be0 2024-05-27T12:35:06.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.619+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] conn(0x7f7ca4074b60 msgr2=0x7f7ca4076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.619+0000 7f7cc8800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6817/3234217179,v1:172.21.15.97:6818/3234217179] conn(0x7f7ca4074b60 0x7f7ca4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.619+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] conn(0x7f3858074be0 msgr2=0x7f3858077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.619+0000 7f387e200640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6800/3824367189,v1:172.21.15.97:6801/3824367189] conn(0x7f3858074be0 0x7f3858077000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f385806c9c0 msgr2=0x7f385806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7ca406c940 msgr2=0x7f7ca406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7ca406c940 0x7f7ca406ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4007a60 tx=0x7f7cb4008040 comp rx=0 tx=0).stop 2024-05-27T12:35:06.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc015d150 msgr2=0x7f7cc0166fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 --2- 172.21.15.94:0/823499952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cc015d150 0x7f7cc0166fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc030000 tx=0x7f7cbc004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:06.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f385806c9c0 0x7f385806ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38781fc040 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-05-27T12:35:06.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc6800640 1 -- 172.21.15.94:0/823499952 reap_dead start 2024-05-27T12:35:06.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 msgr2=0x7f387815c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 --2- 172.21.15.94:0/1405768430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3878162bb0 0x7f387815c0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f38680046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f3877e00640 1 -- 172.21.15.94:0/1405768430 reap_dead start 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 shutdown_connections 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 >> 172.21.15.94:0/1405768430 conn(0x7f38780afc50 msgr2=0x7f38780cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 shutdown_connections 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 >> 172.21.15.94:0/823499952 conn(0x7f7cc00b2660 msgr2=0x7f7cc00d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 shutdown_connections 2024-05-27T12:35:06.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.620+0000 7f387e200640 1 -- 172.21.15.94:0/1405768430 wait complete. 2024-05-27T12:35:06.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.621+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 shutdown_connections 2024-05-27T12:35:06.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.621+0000 7f7cc8800640 1 -- 172.21.15.94:0/823499952 wait complete. 2024-05-27T12:35:06.644 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T12:35:06.645 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T12:35:06.646 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.646 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T12:35:06.647 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T12:35:06.648 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T12:35:06.649 INFO:teuthology.orchestra.run.smithi094.stdout:55834574854 2024-05-27T12:35:06.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T12:35:06.650 INFO:teuthology.orchestra.run.smithi094.stdout:55834574852 2024-05-27T12:35:06.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T12:35:06.651 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:06.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T12:35:06.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.961+0000 7f9ead200640 1 Processor -- start 2024-05-27T12:35:06.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.962+0000 7f9ead200640 1 -- start start 2024-05-27T12:35:06.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.964+0000 7f9ead200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ea8072370 0x7f9ea814b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.964+0000 7f9ead200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ea814bb30 0x7f9ea80ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ea6a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ea8072370 0x7f9ea814b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ea6a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9ea8072370 0x7f9ea814b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56580/0 (socket says 172.21.15.94:56580) 2024-05-27T12:35:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ea6a00640 1 -- 172.21.15.94:0/1346209310 learned_addr learned my addr 172.21.15.94:0/1346209310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ead200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80b3890 con 0x7f9ea8071f90 2024-05-27T12:35:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea814bfa0 con 0x7f9ea8072370 2024-05-27T12:35:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.965+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80b3b70 con 0x7f9ea814bb30 2024-05-27T12:35:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.966+0000 7f9ea6a00640 1 -- 172.21.15.94:0/1346209310 >> v2:172.21.15.97:3300/0 conn(0x7f9ea814bb30 msgr2=0x7f9ea80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.966+0000 7f9ea6a00640 1 --2- 172.21.15.94:0/1346209310 >> v2:172.21.15.97:3300/0 conn(0x7f9ea814bb30 0x7f9ea80ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.966+0000 7f9ea6a00640 1 -- 172.21.15.94:0/1346209310 >> v1:172.21.15.97:6789/0 conn(0x7f9ea8071f90 legacy=0x7f9ea80cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.966+0000 7f9ea6a00640 1 -- 172.21.15.94:0/1346209310 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea80b3e50 con 0x7f9ea8072370 2024-05-27T12:35:06.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.967+0000 7f9ea6a00640 1 --2- 172.21.15.94:0/1346209310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8072370 0x7f9ea814b5f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80cdd80 tx=0x7f9e9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08280554d68a20e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.968+0000 7f9ea6000640 1 -- 172.21.15.94:0/1346209310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98037070 con 0x7f9ea8072370 2024-05-27T12:35:06.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.969+0000 7f9ea6000640 1 -- 172.21.15.94:0/1346209310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e98032440 con 0x7f9ea8072370 2024-05-27T12:35:06.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.969+0000 7f4021400640 1 Processor -- start 2024-05-27T12:35:06.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f4021400640 1 -- start start 2024-05-27T12:35:06.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8072370 msgr2=0x7f9ea814b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 --2- 172.21.15.94:0/1346209310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8072370 0x7f9ea814b5f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80cdd80 tx=0x7f9e9802a210 comp rx=0 tx=0).stop 2024-05-27T12:35:06.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f5836c00640 1 Processor -- start 2024-05-27T12:35:06.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 shutdown_connections 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 --2- 172.21.15.94:0/1346209310 >> v2:172.21.15.97:3300/0 conn(0x7f9ea814bb30 0x7f9ea80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 --2- 172.21.15.94:0/1346209310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8072370 0x7f9ea814b5f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.970+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 >> 172.21.15.94:0/1346209310 conn(0x7f9ea80b5e50 msgr2=0x7f9ea80b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.971+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 shutdown_connections 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.971+0000 7f9ead200640 1 -- 172.21.15.94:0/1346209310 wait complete. 2024-05-27T12:35:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.971+0000 7f5836c00640 1 -- start start 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.972+0000 7f4021400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f401c15e930 0x7f401c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.972+0000 7f4021400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.972+0000 7f4021400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401c15ed10 con 0x7f401c06fd20 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.972+0000 7f4021400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f401c072930 con 0x7f401c15e930 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.972+0000 7f4021400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f401c072000 con 0x7f401c0adb10 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.973+0000 7f401a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f401c15e930 0x7f401c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.973+0000 7f4019e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.973+0000 7f4019e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57974/0 (socket says 172.21.15.94:57974) 2024-05-27T12:35:06.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.973+0000 7f4019e00640 1 -- 172.21.15.94:0/685767348 learned_addr learned my addr 172.21.15.94:0/685767348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.973+0000 7f401b200640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f401c06fd20 0x7f401c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48106/0 (socket says 172.21.15.94:48106) 2024-05-27T12:35:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f9ead200640 1 Processor -- start 2024-05-27T12:35:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f5836c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f583015eb00 0x7f58300cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f5836c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f583015eee0 con 0x7f58300d1f40 2024-05-27T12:35:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7ff245600640 1 Processor -- start 2024-05-27T12:35:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f9ead200640 1 -- start start 2024-05-27T12:35:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f5836c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58300d4660 con 0x7f58300d3e50 2024-05-27T12:35:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7f5836c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f58300d5700 con 0x7f583015eb00 2024-05-27T12:35:06.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.974+0000 7ff245600640 1 -- start start 2024-05-27T12:35:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.975+0000 7f4019e00640 1 -- 172.21.15.94:0/685767348 >> v2:172.21.15.97:3300/0 conn(0x7f401c15e930 msgr2=0x7f401c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.975+0000 7f4019400640 1 -- 172.21.15.94:0/685767348 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1986057404 0 0) 0x7f401c15ed10 con 0x7f401c06fd20 2024-05-27T12:35:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.975+0000 7f4019e00640 1 --2- 172.21.15.94:0/685767348 >> v2:172.21.15.97:3300/0 conn(0x7f401c15e930 0x7f401c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.975+0000 7f4019e00640 1 -- 172.21.15.94:0/685767348 >> v1:172.21.15.94:6789/0 conn(0x7f401c06fd20 legacy=0x7f401c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.975+0000 7f4019e00640 1 -- 172.21.15.94:0/685767348 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c0722e0 con 0x7f401c0adb10 2024-05-27T12:35:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7f9ead200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7f9ead200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 0x7f9ea8146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7f9ead200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 0x7f9ea8146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7ff245600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff238006db0 0x7ff2380ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7ff245600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2380e36e0 con 0x7ff2380061f0 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7ff245600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff238057b40 con 0x7ff2380e3300 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7ff245600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff238057e20 con 0x7ff238006db0 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7ff23f400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff2380061f0 0x7ff2380066a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48112/0 (socket says 172.21.15.94:48112) 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7ff23f400640 1 -- 172.21.15.94:0/1110438277 learned_addr learned my addr 172.21.15.94:0/1110438277 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.976+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 0x7f9ea8146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 0x7f9ea8146200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:57982/0 (socket says 172.21.15.94:57982) 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ea7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ea7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59200/0 (socket says 172.21.15.94:59200) 2024-05-27T12:35:06.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ead200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80ccf60 con 0x7f9ea814bb30 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ead200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80b4410 con 0x7f9ea8071f90 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea80b3540 con 0x7f9ea8072370 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f9ea6a00640 1 -- 172.21.15.94:0/1961281199 learned_addr learned my addr 172.21.15.94:0/1961281199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f582fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f583015eb00 0x7f58300cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f582fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f583015eb00 0x7f58300cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59192/0 (socket says 172.21.15.94:59192) 2024-05-27T12:35:06.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7f582fe00640 1 -- 172.21.15.94:0/2513660943 learned_addr learned my addr 172.21.15.94:0/2513660943 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.977+0000 7ff244c00640 1 --2- 172.21.15.94:0/1110438277 >> v2:172.21.15.97:3300/0 conn(0x7ff238006db0 0x7ff2380ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f9ea7e00640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 0x7f9ea8146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f4019e00640 1 --2- 172.21.15.94:0/685767348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4008009870 tx=0x7f400802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f69e5e4c54dd78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f9ea7400640 1 -- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 msgr2=0x7f9ea8146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f9ea7400640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 0x7f9ea8146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f9ea7400640 1 -- 172.21.15.94:0/1961281199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 msgr2=0x7f9ea8146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238634522 0 0) 0x7f583015eee0 con 0x7f58300d1f40 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5814003520 con 0x7f58300d1f40 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7ff23ea00640 1 -- 172.21.15.94:0/1110438277 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820081846 0 0) 0x7ff2380e36e0 con 0x7ff2380061f0 2024-05-27T12:35:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7ff23ea00640 1 -- 172.21.15.94:0/1110438277 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff228003520 con 0x7ff2380061f0 2024-05-27T12:35:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3756805607 0 0) 0x7f5814003520 con 0x7f58300d1f40 2024-05-27T12:35:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 >> v2:172.21.15.97:3300/0 conn(0x7f583015eb00 msgr2=0x7f58300cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 --2- 172.21.15.94:0/2513660943 >> v2:172.21.15.97:3300/0 conn(0x7f583015eb00 0x7f58300cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.978+0000 7f9ea7400640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 0x7f9ea8146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f9ea7400640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea8144f60 con 0x7f9ea8071f90 2024-05-27T12:35:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 >> v1:172.21.15.97:6789/0 conn(0x7f58300d3e50 legacy=0x7f58300d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300d4940 con 0x7f58300d1f40 2024-05-27T12:35:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f5818001650 con 0x7f58300d1f40 2024-05-27T12:35:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 >> v1:172.21.15.94:6789/0 conn(0x7f58300d1f40 legacy=0x7f58300b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58140039b0 0x7f5814003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5814004800 0x7f5814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 0x7f581400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58300d5700 con 0x7f58140051c0 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f581400c010 con 0x7f5814004800 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f581400c210 con 0x7f58140039b0 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f4019400640 1 -- 172.21.15.94:0/685767348 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008037070 con 0x7f401c0adb10 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.980+0000 7f4019400640 1 -- 172.21.15.94:0/685767348 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4008032440 con 0x7f401c0adb10 2024-05-27T12:35:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.980+0000 7f4019400640 1 -- 172.21.15.94:0/685767348 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008032810 con 0x7f401c0adb10 2024-05-27T12:35:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7f9ea7e00640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 0x7f9ea8146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7ff244c00640 1 -- 172.21.15.94:0/1110438277 >> v1:172.21.15.97:6789/0 conn(0x7ff2380e3300 legacy=0x7ff2380572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7ff244c00640 1 -- 172.21.15.94:0/1110438277 >> v1:172.21.15.94:6789/0 conn(0x7ff2380061f0 legacy=0x7ff2380066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.979+0000 7ff244c00640 1 -- 172.21.15.94:0/1110438277 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff238058100 con 0x7ff238006db0 2024-05-27T12:35:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.980+0000 7f5834c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 0x7f581400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7ff3b7200640 1 Processor -- start 2024-05-27T12:35:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7ff244c00640 1 --2- 172.21.15.94:0/1110438277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff2380ede10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2340097a0 tx=0x7ff23402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0b313a0b6d4628 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7ff3b7200640 1 -- start start 2024-05-27T12:35:06.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f582f400640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5814004800 0x7f5814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f582fe00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58140039b0 0x7f5814003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 -- 172.21.15.94:0/685767348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 msgr2=0x7f401c0b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 --2- 172.21.15.94:0/685767348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4008009870 tx=0x7f400802a270 comp rx=0 tx=0).stop 2024-05-27T12:35:06.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 -- 172.21.15.94:0/685767348 shutdown_connections 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 --2- 172.21.15.94:0/685767348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c0adb10 0x7f401c0b4540 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 --2- 172.21.15.94:0/685767348 >> v2:172.21.15.97:3300/0 conn(0x7f401c15e930 0x7f401c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f4021400640 1 -- 172.21.15.94:0/685767348 >> 172.21.15.94:0/685767348 conn(0x7f401c0b2240 msgr2=0x7f401c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.981+0000 7f9ea7400640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c700 tx=0x7f9ea000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f5834c00640 1 -- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58140039b0 msgr2=0x7f5814003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f5834c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58140039b0 0x7f5814003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f4021400640 1 -- 172.21.15.94:0/685767348 shutdown_connections 2024-05-27T12:35:06.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f5834c00640 1 -- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5814004800 msgr2=0x7f5814004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f5834c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5814004800 0x7f5814004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f4021400640 1 -- 172.21.15.94:0/685767348 wait complete. 2024-05-27T12:35:06.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.982+0000 7f5834c00640 1 -- 172.21.15.94:0/2513660943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300d4940 con 0x7f58140051c0 2024-05-27T12:35:06.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7ff23ea00640 1 -- 172.21.15.94:0/1110438277 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234037070 con 0x7ff238006db0 2024-05-27T12:35:06.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7ff23ea00640 1 -- 172.21.15.94:0/1110438277 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234032440 con 0x7ff238006db0 2024-05-27T12:35:06.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7ff23ea00640 1 -- 172.21.15.94:0/1110438277 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234032850 con 0x7ff238006db0 2024-05-27T12:35:06.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0003b70 con 0x7f9ea8071f90 2024-05-27T12:35:06.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7ff245600640 1 -- 172.21.15.94:0/1110438277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 msgr2=0x7ff2380ede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7ff245600640 1 --2- 172.21.15.94:0/1110438277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff2380ede10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2340097a0 tx=0x7ff23402a1c0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea8163490 con 0x7f9ea8071f90 2024-05-27T12:35:06.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.983+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea8163a50 con 0x7f9ea8071f90 2024-05-27T12:35:06.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7f1b4fe00640 1 Processor -- start 2024-05-27T12:35:06.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7ff3b7200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3a80df3d0 0x7ff3a804fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7ff245600640 1 -- 172.21.15.94:0/1110438277 shutdown_connections 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7ff245600640 1 --2- 172.21.15.94:0/1110438277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff2380ede10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7ff245600640 1 -- 172.21.15.94:0/1110438277 >> 172.21.15.94:0/1110438277 conn(0x7ff238058ef0 msgr2=0x7ff238059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7ff3b7200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a8050320 0x7ff3a80530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3b7200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a8054d30 con 0x7ff3a80deff0 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3af400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3a80df3d0 0x7ff3a804fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3b7200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a8050790 con 0x7ff3a80df3d0 2024-05-27T12:35:06.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3b7200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a8054630 con 0x7ff3a8050320 2024-05-27T12:35:06.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7f1b4fe00640 1 -- start start 2024-05-27T12:35:06.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.984+0000 7f5834c00640 1 -- 172.21.15.94:0/2513660943 reap_dead start 2024-05-27T12:35:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3af400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3a80df3d0 0x7ff3a804fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59236/0 (socket says 172.21.15.94:59236) 2024-05-27T12:35:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3af400640 1 -- 172.21.15.94:0/2389984385 learned_addr learned my addr 172.21.15.94:0/2389984385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff3b5200640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a8050320 0x7ff3a80530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7ff3aea00640 1 -- 172.21.15.94:0/2389984385 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169348980 0 0) 0x7ff3a8054d30 con 0x7ff3a80deff0 2024-05-27T12:35:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f4021400640 1 Processor -- start 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7ff3af400640 1 -- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a8050320 msgr2=0x7ff3a80530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7ff3af400640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a8050320 0x7ff3a80530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7ff3af400640 1 -- 172.21.15.94:0/2389984385 >> v1:172.21.15.94:6789/0 conn(0x7ff3a80deff0 legacy=0x7ff3a804f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7ff3af400640 1 -- 172.21.15.94:0/2389984385 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8006dd0 con 0x7ff3a80df3d0 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f5834c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 0x7f581400b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5818007fd0 tx=0x7f5814005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75780f4c64b6f13 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818007bb0 con 0x7f58140051c0 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5818005290 con 0x7f58140051c0 2024-05-27T12:35:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f582ea00640 1 -- 172.21.15.94:0/2513660943 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818003830 con 0x7f58140051c0 2024-05-27T12:35:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f4021400640 1 -- start start 2024-05-27T12:35:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f1b4fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 0x7f1b50167380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 0x7f1b500b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f1b4fe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b500b57f0 con 0x7f1b5006f3a0 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f1b4fe00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f1b50166ef0 con 0x7f1b50162d30 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.986+0000 7f1b4fe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1b500b5ad0 con 0x7f1b500afe60 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f1b4ea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 0x7f1b50167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f1b4ea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 0x7f1b50167380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56612/0 (socket says 172.21.15.94:56612) 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f1b4ea00640 1 -- 172.21.15.94:0/4080204232 learned_addr learned my addr 172.21.15.94:0/4080204232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f1b4e000640 1 --2- 172.21.15.94:0/4080204232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 0x7f1b500b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f4021400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f4021400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401c0adb10 0x7f401c13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f4021400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 0x7f401c15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f4021400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b3850 con 0x7f401c15e930 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f4021400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b36d0 con 0x7f401c0adb10 2024-05-27T12:35:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f4021400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f401c0afbf0 con 0x7f401c06fd20 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff245600640 1 -- 172.21.15.94:0/1110438277 shutdown_connections 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.985+0000 7ff245600640 1 -- 172.21.15.94:0/1110438277 wait complete. 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7ff245600640 1 Processor -- start 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7ff245600640 1 -- start start 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f1b4e000640 1 -- 172.21.15.94:0/4080204232 >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 msgr2=0x7f1b50167380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f1b4e000640 1 --2- 172.21.15.94:0/4080204232 >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 0x7f1b50167380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f1b4e000640 1 -- 172.21.15.94:0/4080204232 >> v1:172.21.15.97:6789/0 conn(0x7f1b5006f3a0 legacy=0x7f1b500b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f1b4e000640 1 -- 172.21.15.94:0/4080204232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b501668a0 con 0x7f1b500afe60 2024-05-27T12:35:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58026/0 (socket says 172.21.15.94:58026) 2024-05-27T12:35:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 -- 172.21.15.94:0/913030232 learned_addr learned my addr 172.21.15.94:0/913030232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7ff3af400640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80df3d0 0x7ff3a804fde0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a402c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a8924e519a8723 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401b200640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 0x7f401c15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f4019e00640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401c0adb10 0x7f401c13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea8075460 con 0x7f9ea8071f90 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f5836c00640 1 -- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 msgr2=0x7f581400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.987+0000 7f5836c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 0x7f581400b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5818007fd0 tx=0x7f5814005830 comp rx=0 tx=0).stop 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f5836c00640 1 -- 172.21.15.94:0/2513660943 shutdown_connections 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f5836c00640 1 --2- 172.21.15.94:0/2513660943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58140051c0 0x7f581400b9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f5836c00640 1 -- 172.21.15.94:0/2513660943 >> 172.21.15.94:0/2513660943 conn(0x7f5830067a50 msgr2=0x7f5830067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f5836c00640 1 -- 172.21.15.94:0/2513660943 shutdown_connections 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.988+0000 7f5836c00640 1 -- 172.21.15.94:0/2513660943 wait complete. 2024-05-27T12:35:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 0x7ff2380d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 -- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401c0adb10 msgr2=0x7f401c13fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401c0adb10 0x7f401c13fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.989+0000 7f401a800640 1 -- 172.21.15.94:0/913030232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 msgr2=0x7f401c15ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7f1b4e000640 1 --2- 172.21.15.94:0/4080204232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 0x7f1b500b4f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0097a0 tx=0x7f1b3c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f993fb1ede4d11b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff23804da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 0x7ff23803ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7f401a800640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 0x7f401c15ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7f401a800640 1 -- 172.21.15.94:0/913030232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c0ce420 con 0x7f401c06fd20 2024-05-27T12:35:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7f401b200640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 0x7f401c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff238058be0 con 0x7ff2380e3300 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff23803d130 con 0x7ff238006db0 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff245600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff23804c9f0 con 0x7ff2380061f0 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 0x7ff2380d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.990+0000 7ff244c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 0x7ff23803ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 0x7ff2380d9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58036/0 (socket says 172.21.15.94:58036) 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff23fe00640 1 -- 172.21.15.94:0/3251782741 learned_addr learned my addr 172.21.15.94:0/3251782741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f7890200640 1 Processor -- start 2024-05-27T12:35:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff23f400640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff23804da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f401a800640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f400c00d5f0 tx=0x7f400c00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f7890200640 1 -- start start 2024-05-27T12:35:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f2096c00640 1 Processor -- start 2024-05-27T12:35:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f5836c00640 1 Processor -- start 2024-05-27T12:35:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7f2096c00640 1 -- start start 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff244c00640 1 -- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 msgr2=0x7ff2380d9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff244c00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 0x7ff2380d9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff244c00640 1 -- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 msgr2=0x7ff23804da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.991+0000 7ff244c00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff23804da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7ff244c00640 1 -- 172.21.15.94:0/3251782741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23804ccd0 con 0x7ff2380e3300 2024-05-27T12:35:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea0003d10 con 0x7f9ea8071f90 2024-05-27T12:35:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7f5836c00640 1 -- start start 2024-05-27T12:35:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7ff3aea00640 1 -- 172.21.15.94:0/2389984385 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4038070 con 0x7ff3a80df3d0 2024-05-27T12:35:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7f1b4d600640 1 -- 172.21.15.94:0/4080204232 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c037070 con 0x7f1b500afe60 2024-05-27T12:35:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f7890200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.992+0000 7ff244c00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 0x7ff23803ae00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff234031000 tx=0x7ff2340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f7890200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f78880b4fb0 0x7f788815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7ff3aea00640 1 -- 172.21.15.94:0/2389984385 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a4031710 con 0x7ff3a80df3d0 2024-05-27T12:35:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7ff3aea00640 1 -- 172.21.15.94:0/2389984385 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4031ae0 con 0x7ff3a80df3d0 2024-05-27T12:35:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0006d90 con 0x7f9ea8071f90 2024-05-27T12:35:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f1b4d600640 1 -- 172.21.15.94:0/4080204232 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c032440 con 0x7f1b500afe60 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f7890200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788815b840 con 0x7f788815af50 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f7890200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f788815e980 con 0x7f78880b4fb0 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f7890200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f78880d02d0 con 0x7f788806b930 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58300c8390 0x7f58300c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58300ccc90 0x7f58300cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58300d51e0 con 0x7f58300ccc90 2024-05-27T12:35:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58300d4f00 con 0x7f58140051c0 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f5836c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f58300d42c0 con 0x7f58300c8390 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f788d800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0125f0 con 0x7f401c06fd20 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c019070 con 0x7f401c06fd20 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0129d0 con 0x7f401c06fd20 2024-05-27T12:35:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c0ce700 con 0x7f401c06fd20 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f2096c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f2088055a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f2096c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2088038da0 0x7f2088054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f2096c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20880553e0 con 0x7f20880e43f0 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f2096c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2088056500 con 0x7f2088038da0 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.993+0000 7f2096c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f20880556c0 con 0x7f20880389c0 2024-05-27T12:35:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c14ee10 con 0x7f401c06fd20 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f1b4d600640 1 -- 172.21.15.94:0/4080204232 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c032810 con 0x7f1b500afe60 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7ff3b7200640 1 -- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80df3d0 msgr2=0x7ff3a804fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7ff3b7200640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80df3d0 0x7ff3a804fde0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a402c520 comp rx=0 tx=0).stop 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 -- 172.21.15.94:0/2389984385 shutdown_connections 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a8050320 0x7ff3a80530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 --2- 172.21.15.94:0/2389984385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80df3d0 0x7ff3a804fde0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 -- 172.21.15.94:0/2389984385 >> 172.21.15.94:0/2389984385 conn(0x7ff3a800a3f0 msgr2=0x7ff3a800a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f5834c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58300ccc90 0x7f58300cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f582fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f582fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59280/0 (socket says 172.21.15.94:59280) 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f582fe00640 1 -- 172.21.15.94:0/2986089377 learned_addr learned my addr 172.21.15.94:0/2986089377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f582f400640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58300c8390 0x7f58300c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f582fe00640 1 -- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58300c8390 msgr2=0x7f58300c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f582fe00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58300c8390 0x7f58300c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f582fe00640 1 -- 172.21.15.94:0/2986089377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58300ccc90 msgr2=0x7f58300cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f582fe00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58300ccc90 0x7f58300cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f582fe00640 1 -- 172.21.15.94:0/2986089377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300ce2a0 con 0x7f58140051c0 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234037050 con 0x7ff2380e3300 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234043070 con 0x7ff2380e3300 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23403cb10 con 0x7ff2380e3300 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f1b4fe00640 1 -- 172.21.15.94:0/4080204232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 msgr2=0x7f1b500b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/4080204232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 0x7f1b500b4f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0097a0 tx=0x7f1b3c02a180 comp rx=0 tx=0).stop 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 -- 172.21.15.94:0/4080204232 shutdown_connections 2024-05-27T12:35:07.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/4080204232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b500afe60 0x7f1b500b4f50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/4080204232 >> v2:172.21.15.94:3300/0 conn(0x7f1b50162d30 0x7f1b50167380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 -- 172.21.15.94:0/4080204232 >> 172.21.15.94:0/4080204232 conn(0x7f1b500b25c0 msgr2=0x7f1b500b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 -- 172.21.15.94:0/4080204232 shutdown_connections 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f1b4fe00640 1 -- 172.21.15.94:0/4080204232 wait complete. 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f2094e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2088038da0 0x7f2088054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f2094e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2088038da0 0x7f2088054270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59270/0 (socket says 172.21.15.94:59270) 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f2094e00640 1 -- 172.21.15.94:0/3011152453 learned_addr learned my addr 172.21.15.94:0/3011152453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f788d800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58050/0 (socket says 172.21.15.94:58050) 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f788d800640 1 -- 172.21.15.94:0/2241648004 learned_addr learned my addr 172.21.15.94:0/2241648004 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f788ce00640 1 --2- 172.21.15.94:0/2241648004 >> v2:172.21.15.94:3300/0 conn(0x7f78880b4fb0 0x7f788815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.994+0000 7f2095800640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f2088055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f2094e00640 1 -- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 msgr2=0x7f2088055a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f2094e00640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f2088055a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f2094e00640 1 -- 172.21.15.94:0/3011152453 >> v1:172.21.15.97:6789/0 conn(0x7f20880e43f0 legacy=0x7f20880516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f2094e00640 1 -- 172.21.15.94:0/3011152453 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2088056750 con 0x7f2088038da0 2024-05-27T12:35:07.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 -- 172.21.15.94:0/2389984385 shutdown_connections 2024-05-27T12:35:07.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7ff3b7200640 1 -- 172.21.15.94:0/2389984385 wait complete. 2024-05-27T12:35:07.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7f582fe00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5828009fb0 tx=0x7f582800f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7ff3b7200640 1 Processor -- start 2024-05-27T12:35:07.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f788d800640 1 -- 172.21.15.94:0/2241648004 >> v2:172.21.15.94:3300/0 conn(0x7f78880b4fb0 msgr2=0x7f788815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f788d800640 1 --2- 172.21.15.94:0/2241648004 >> v2:172.21.15.94:3300/0 conn(0x7f78880b4fb0 0x7f788815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f788d800640 1 -- 172.21.15.94:0/2241648004 >> v1:172.21.15.94:6789/0 conn(0x7f788815af50 legacy=0x7f7888071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7ff245600640 1 -- 172.21.15.94:0/3251782741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23804cfb0 con 0x7ff2380e3300 2024-05-27T12:35:07.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7ff245600640 1 -- 172.21.15.94:0/3251782741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23804d570 con 0x7ff2380e3300 2024-05-27T12:35:07.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.996+0000 7f788d800640 1 -- 172.21.15.94:0/2241648004 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788815bb20 con 0x7f788806b930 2024-05-27T12:35:07.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.995+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c068a70 con 0x7f401c06fd20 2024-05-27T12:35:07.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7ff3b7200640 1 -- start start 2024-05-27T12:35:07.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.997+0000 7f788d800640 1 --2- 172.21.15.94:0/2241648004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c02c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0f32d3ed67ef36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7f1b4fe00640 1 Processor -- start 2024-05-27T12:35:07.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7f1b4fe00640 1 -- start start 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ea00050d0 con 0x7f9ea8071f90 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7ff3b7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 0x7ff3a80dd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7ff3b7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80deff0 0x7ff3a8175850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b7200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 0x7ff3a804c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8007670 con 0x7ff3a8050320 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8007390 con 0x7ff3a80deff0 2024-05-27T12:35:07.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b7200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a8007b90 con 0x7ff3a80df3d0 2024-05-27T12:35:07.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f2094e00640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2088038da0 0x7f2088054270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f207c009870 tx=0x7f207c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddd379855657bde server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b5006f3a0 0x7f1b50165ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b500afe60 0x7f1b501598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.998+0000 7ff245600640 1 -- 172.21.15.94:0/3251782741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2380e4100 con 0x7ff2380e3300 2024-05-27T12:35:07.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 0x7ff3a80dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b500b5490 con 0x7f1b50162d30 2024-05-27T12:35:07.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50163110 con 0x7f1b500afe60 2024-05-27T12:35:07.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f1b4fe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5006f830 con 0x7f1b5006f3a0 2024-05-27T12:35:07.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b5200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 0x7ff3a804c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b5200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 0x7ff3a804c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58080/0 (socket says 172.21.15.94:58080) 2024-05-27T12:35:07.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3b5200640 1 -- 172.21.15.94:0/3726724674 learned_addr learned my addr 172.21.15.94:0/3726724674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7f9e9f400640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 0x7f9e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:06.999+0000 7ff3af400640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80deff0 0x7ff3a8175850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56664/0 (socket says 172.21.15.94:56664) 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 -- 172.21.15.94:0/2653872713 learned_addr learned my addr 172.21.15.94:0/2653872713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4ea00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b5006f3a0 0x7f1b50165ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4e000640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b500afe60 0x7f1b501598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea008c180 con 0x7f9ea8071f90 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff3afe00640 1 -- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 msgr2=0x7ff3a804c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff3afe00640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 0x7ff3a804c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff3afe00640 1 -- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80deff0 msgr2=0x7ff3a8175850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff3afe00640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80deff0 0x7ff3a8175850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff3afe00640 1 -- 172.21.15.94:0/3726724674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a804e5f0 con 0x7ff3a8050320 2024-05-27T12:35:07.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff23402ad10 con 0x7ff2380e3300 2024-05-27T12:35:07.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f7887e00640 1 -- 172.21.15.94:0/2241648004 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c039070 con 0x7f788806b930 2024-05-27T12:35:07.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f7887e00640 1 -- 172.21.15.94:0/2241648004 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787c034640 con 0x7f788806b930 2024-05-27T12:35:07.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f7887e00640 1 -- 172.21.15.94:0/2241648004 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c034a10 con 0x7f788806b930 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 -- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b5006f3a0 msgr2=0x7f1b50165ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b5006f3a0 0x7f1b50165ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 -- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b500afe60 msgr2=0x7f1b501598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b500afe60 0x7f1b501598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.000+0000 7f1b4f400640 1 -- 172.21.15.94:0/2653872713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b50159ff0 con 0x7f1b50162d30 2024-05-27T12:35:07.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b4f400640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c3c0 tx=0x7f1b4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 -- 172.21.15.94:0/2241648004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 msgr2=0x7f7888151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 --2- 172.21.15.94:0/2241648004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c02c390 comp rx=0 tx=0).stop 2024-05-27T12:35:07.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f9ea6a00640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 0x7f9e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7ff3afe00640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 0x7ff3a80dd260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00690b0 tx=0x7ff3b00694e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400052d0 con 0x7f1b50162d30 2024-05-27T12:35:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40005490 con 0x7f1b50162d30 2024-05-27T12:35:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007260 con 0x7f1b50162d30 2024-05-27T12:35:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0075320 con 0x7ff3a8050320 2024-05-27T12:35:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5015a2d0 con 0x7f1b50162d30 2024-05-27T12:35:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b008f030 con 0x7ff3a8050320 2024-05-27T12:35:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0075700 con 0x7ff3a8050320 2024-05-27T12:35:07.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 -- 172.21.15.94:0/2241648004 shutdown_connections 2024-05-27T12:35:07.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 --2- 172.21.15.94:0/2241648004 >> v2:172.21.15.94:3300/0 conn(0x7f78880b4fb0 0x7f788815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 --2- 172.21.15.94:0/2241648004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f788806b930 0x7f7888151910 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f7890200640 1 -- 172.21.15.94:0/2241648004 >> 172.21.15.94:0/2241648004 conn(0x7f78880b2660 msgr2=0x7f78880b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a804e030 con 0x7ff3a8050320 2024-05-27T12:35:07.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7ff233e00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 0x7ff22006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff234078490 con 0x7ff2380e3300 2024-05-27T12:35:07.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.001+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5015a890 con 0x7f1b50162d30 2024-05-27T12:35:07.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7ff23fe00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 0x7ff22006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7f7890200640 1 -- 172.21.15.94:0/2241648004 shutdown_connections 2024-05-27T12:35:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7f7890200640 1 -- 172.21.15.94:0/2241648004 wait complete. 2024-05-27T12:35:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.003+0000 7ff23fe00640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 0x7ff22006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007fd0 tx=0x7ff22c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50075460 con 0x7f1b50162d30 2024-05-27T12:35:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.002+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a804c050 con 0x7ff3a8050320 2024-05-27T12:35:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8005700 con 0x7ff3a8050320 2024-05-27T12:35:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7f7890200640 1 Processor -- start 2024-05-27T12:35:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f400c012b70 con 0x7f401c06fd20 2024-05-27T12:35:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7f9ea6a00640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 0x7f9e8806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea81ee060 tx=0x7f9e9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.005+0000 7f4012a00640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 0x7f3ffc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.005+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f400c08ca40 con 0x7f401c06fd20 2024-05-27T12:35:07.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.004+0000 7f7890200640 1 -- start start 2024-05-27T12:35:07.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b40007400 con 0x7f1b50162d30 2024-05-27T12:35:07.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 0x7f788814de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78880b4fb0 0x7f7888155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f4019e00640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 0x7f3ffc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f788815be00 con 0x7f788815af50 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7888151e50 con 0x7f788806b930 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f7890200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f788815f350 con 0x7f78880b4fb0 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f788e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f788e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56674/0 (socket says 172.21.15.94:56674) 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f788e200640 1 -- 172.21.15.94:0/2096359153 learned_addr learned my addr 172.21.15.94:0/2096359153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.006+0000 7f788d800640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 0x7f788814de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788ce00640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78880b4fb0 0x7f7888155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 -- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78880b4fb0 msgr2=0x7f7888155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78880b4fb0 0x7f7888155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 -- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 msgr2=0x7f788814de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 0x7f788814de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 -- 172.21.15.94:0/2096359153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788814e550 con 0x7f788815af50 2024-05-27T12:35:07.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788d800640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 0x7f788814de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:07.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f788e200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f788000b970 tx=0x7f788000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f4019e00640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 0x7f3ffc06ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f1b46a00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 0x7f1b3006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.007+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b4008b1b0 con 0x7f1b50162d30 2024-05-27T12:35:07.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.008+0000 7f1b4ea00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 0x7f1b3006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.009+0000 7f1b4ea00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 0x7f1b3006ee30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b50072b10 tx=0x7f1b48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.009+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff2380e4100 con 0x7ff2380e3300 2024-05-27T12:35:07.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.011+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3b00933d0 con 0x7ff3a8050320 2024-05-27T12:35:07.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.011+0000 7ff3a3e00640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 0x7ff39006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.012+0000 7ff3af400640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 0x7ff39006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.013+0000 7ff3af400640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 0x7ff39006ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4035000 tx=0x7ff3a4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.014+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ea0020020 con 0x7f9ea8071f90 2024-05-27T12:35:07.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.015+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b400595c0 con 0x7f1b50162d30 2024-05-27T12:35:07.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.015+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b00d30b0 con 0x7ff3a8050320 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.014+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880019070 con 0x7f788815af50 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.015+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788000cb40 con 0x7f788815af50 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.015+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880014440 con 0x7f788815af50 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.016+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788814e830 con 0x7f788815af50 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.016+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788814edf0 con 0x7f788815af50 2024-05-27T12:35:07.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.017+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7888068a70 con 0x7f788815af50 2024-05-27T12:35:07.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f208fe00640 1 -- 172.21.15.94:0/3011152453 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c037070 con 0x7f2088038da0 2024-05-27T12:35:07.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828019070 con 0x7f58140051c0 2024-05-27T12:35:07.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582800fe90 con 0x7f58140051c0 2024-05-27T12:35:07.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828015650 con 0x7f58140051c0 2024-05-27T12:35:07.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f208fe00640 1 -- 172.21.15.94:0/3011152453 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c032440 con 0x7f2088038da0 2024-05-27T12:35:07.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f208fe00640 1 -- 172.21.15.94:0/3011152453 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c032810 con 0x7f2088038da0 2024-05-27T12:35:07.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.018+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58300cda00 con 0x7f58140051c0 2024-05-27T12:35:07.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.019+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58300cdfc0 con 0x7f58140051c0 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 -- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2088038da0 msgr2=0x7f2088054270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2088038da0 0x7f2088054270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f207c009870 tx=0x7f207c02a180 comp rx=0 tx=0).stop 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 -- 172.21.15.94:0/3011152453 shutdown_connections 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2088038da0 0x7f2088054270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 --2- 172.21.15.94:0/3011152453 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f2088055a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 -- 172.21.15.94:0/3011152453 >> 172.21.15.94:0/3011152453 conn(0x7f2088053750 msgr2=0x7f2088053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5830068a70 con 0x7f58140051c0 2024-05-27T12:35:07.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 -- 172.21.15.94:0/3011152453 shutdown_connections 2024-05-27T12:35:07.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.020+0000 7f2096c00640 1 -- 172.21.15.94:0/3011152453 wait complete. 2024-05-27T12:35:07.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.023+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3b00d5010 con 0x7ff3a8050320 2024-05-27T12:35:07.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.023+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f582801c070 con 0x7f58140051c0 2024-05-27T12:35:07.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.022+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f400c05aee0 con 0x7f401c06fd20 2024-05-27T12:35:07.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.023+0000 7f2096c00640 1 Processor -- start 2024-05-27T12:35:07.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.023+0000 7f2096c00640 1 -- start start 2024-05-27T12:35:07.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f5827e00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 0x7f580c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f582808c710 con 0x7f58140051c0 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f208804edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2088038da0 0x7f20880339f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 0x7f20880e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f208803b520 con 0x7f2088038da0 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f208803b3a0 con 0x7f20880e43f0 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f2096c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f208803b6a0 con 0x7f20880389c0 2024-05-27T12:35:07.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.025+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f788001c070 con 0x7f788815af50 2024-05-27T12:35:07.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.026+0000 7f2095800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f208804edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.026+0000 7f2095800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f208804edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58114/0 (socket says 172.21.15.94:58114) 2024-05-27T12:35:07.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.026+0000 7f2095800640 1 -- 172.21.15.94:0/3017232484 learned_addr learned my addr 172.21.15.94:0/3017232484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.026+0000 7f2096200640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 0x7f20880e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f7885600640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 0x7f786806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f788008b900 con 0x7f788815af50 2024-05-27T12:35:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.028+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78800b9b80 con 0x7f788815af50 2024-05-27T12:35:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.028+0000 7f582f400640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 0x7f580c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.028+0000 7f788d800640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 0x7f786806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 -- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 msgr2=0x7f208804edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f208804edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 -- 172.21.15.94:0/3017232484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2088038da0 msgr2=0x7f20880339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2088038da0 0x7f20880339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 -- 172.21.15.94:0/3017232484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208804f330 con 0x7f20880e43f0 2024-05-27T12:35:07.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.027+0000 7f2096200640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 0x7f20880e3250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2090050200 tx=0x7f20900694e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.030+0000 7f582f400640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 0x7f580c06ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5830144af0 tx=0x7f5820009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.030+0000 7f788d800640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 0x7f786806eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f787c036000 tx=0x7f787c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.036+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090075320 con 0x7f20880e43f0 2024-05-27T12:35:07.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.037+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209008f070 con 0x7f20880e43f0 2024-05-27T12:35:07.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.037+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20880d7d40 con 0x7f20880e43f0 2024-05-27T12:35:07.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.037+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20880d4940 con 0x7f20880e43f0 2024-05-27T12:35:07.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.037+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5828010040 con 0x7f58140051c0 2024-05-27T12:35:07.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.037+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090075700 con 0x7f20880e43f0 2024-05-27T12:35:07.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.040+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c005290 con 0x7f20880e43f0 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.041+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2090094470 con 0x7f20880e43f0 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.043+0000 7f208d600640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 0x7f207006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.043+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2090106c00 con 0x7f20880e43f0 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.044+0000 7f2095800640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 0x7f207006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.045+0000 7f2095800640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 0x7f207006ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2084009b60 tx=0x7f2084009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.056+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20900d5010 con 0x7f20880e43f0 2024-05-27T12:35:07.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.414+0000 7ff245600640 1 -- 172.21.15.94:0/3251782741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff2380382b0 con 0x7ff2380e3300 2024-05-27T12:35:07.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.415+0000 7ff233e00640 1 -- 172.21.15.94:0/3251782741 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2380382b0 con 0x7ff2380e3300 2024-05-27T12:35:07.418 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.423+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 msgr2=0x7ff22006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.423+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 0x7ff22006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007fd0 tx=0x7ff22c009040 comp rx=0 tx=0).stop 2024-05-27T12:35:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.423+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 msgr2=0x7ff23803ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.423+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 0x7ff23803ae00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff234031000 tx=0x7ff2340046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 shutdown_connections 2024-05-27T12:35:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff22006c940 0x7ff22006ee00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff238006db0 0x7ff23804da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2380e3300 0x7ff23803ae00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 --2- 172.21.15.94:0/3251782741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380061f0 0x7ff2380d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.424+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 >> 172.21.15.94:0/3251782741 conn(0x7ff238058ef0 msgr2=0x7ff2380e39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.425+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 shutdown_connections 2024-05-27T12:35:07.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.425+0000 7ff231600640 1 -- 172.21.15.94:0/3251782741 wait complete. 2024-05-27T12:35:07.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.445+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9ea814c8a0 con 0x7f9ea8071f90 2024-05-27T12:35:07.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.447+0000 7f9e9f400640 1 -- 172.21.15.94:0/1961281199 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ea005a5a0 con 0x7f9ea8071f90 2024-05-27T12:35:07.448 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.451+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 msgr2=0x7f9e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.451+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 0x7f9e8806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea81ee060 tx=0x7f9e9802e000 comp rx=0 tx=0).stop 2024-05-27T12:35:07.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.451+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 msgr2=0x7f9ea8144a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.451+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c700 tx=0x7f9ea000cbc0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-05-27T12:35:07.452 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 shutdown_connections 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e8806c8f0 0x7f9e8806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea814bb30 0x7f9ea8146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ea8072370 0x7f9ea8146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 --2- 172.21.15.94:0/1961281199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ea8071f90 0x7f9ea8144a20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 >> 172.21.15.94:0/1961281199 conn(0x7f9ea80b5e50 msgr2=0x7f9ea806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 shutdown_connections 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.452+0000 7f9ead200640 1 -- 172.21.15.94:0/1961281199 wait complete. 2024-05-27T12:35:07.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.454+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1b50073380 con 0x7f1b50162d30 2024-05-27T12:35:07.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.455+0000 7f1b46a00640 1 -- 172.21.15.94:0/2653872713 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b40058ec0 con 0x7f1b50162d30 2024-05-27T12:35:07.454 INFO:teuthology.orchestra.run.smithi094.stdout:55834574854 2024-05-27T12:35:07.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.457+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f401c06e5d0 con 0x7f401c06fd20 2024-05-27T12:35:07.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.458+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 msgr2=0x7f1b3006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.458+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 0x7f1b3006ee30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b50072b10 tx=0x7f1b48009040 comp rx=0 tx=0).stop 2024-05-27T12:35:07.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.458+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 msgr2=0x7f1b50204e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.458+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c3c0 tx=0x7f1b4000c880 comp rx=0 tx=0).stop 2024-05-27T12:35:07.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.458+0000 7f4012a00640 1 -- 172.21.15.94:0/913030232 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f400c05a7e0 con 0x7f401c06fd20 2024-05-27T12:35:07.458 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 shutdown_connections 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1b3006c970 0x7f1b3006ee30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b500afe60 0x7f1b501598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b50162d30 0x7f1b50204e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 --2- 172.21.15.94:0/2653872713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b5006f3a0 0x7f1b50165ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 >> 172.21.15.94:0/2653872713 conn(0x7f1b500b25c0 msgr2=0x7f1b500cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.459+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 shutdown_connections 2024-05-27T12:35:07.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.460+0000 7f1b4fe00640 1 -- 172.21.15.94:0/2653872713 wait complete. 2024-05-27T12:35:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 msgr2=0x7f3ffc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 0x7f3ffc06ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).stop 2024-05-27T12:35:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 msgr2=0x7f401c13b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f400c00d5f0 tx=0x7f400c00dab0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 shutdown_connections 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3ffc06c9c0 0x7f3ffc06ee80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401c0adb10 0x7f401c13fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f401c15e930 0x7f401c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 --2- 172.21.15.94:0/913030232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f401c06fd20 0x7f401c13b500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.462+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 >> 172.21.15.94:0/913030232 conn(0x7f401c0b2240 msgr2=0x7f401c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.463+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 shutdown_connections 2024-05-27T12:35:07.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.463+0000 7f4021400640 1 -- 172.21.15.94:0/913030232 wait complete. 2024-05-27T12:35:07.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.464+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff3a8054b10 con 0x7ff3a8050320 2024-05-27T12:35:07.464 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.465+0000 7ff3a3e00640 1 -- 172.21.15.94:0/3726724674 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3b00d9310 con 0x7ff3a8050320 2024-05-27T12:35:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.465+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7888152810 con 0x7f788815af50 2024-05-27T12:35:07.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.466+0000 7f7885600640 1 -- 172.21.15.94:0/2096359153 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f788008b090 con 0x7f788815af50 2024-05-27T12:35:07.466 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 msgr2=0x7ff39006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f206c005580 con 0x7f20880e43f0 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 0x7ff39006ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4035000 tx=0x7ff3a4008030 comp rx=0 tx=0).stop 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 msgr2=0x7ff3a80dd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 0x7ff3a80dd260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00690b0 tx=0x7ff3b00694e0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 shutdown_connections 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff39006c940 0x7ff39006ee00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3a80deff0 0x7ff3a8175850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3a80df3d0 0x7ff3a804c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 --2- 172.21.15.94:0/3726724674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3a8050320 0x7ff3a80dd260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 >> 172.21.15.94:0/3726724674 conn(0x7ff3a800a3f0 msgr2=0x7ff3a8049c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.469+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 shutdown_connections 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7ff3b7200640 1 -- 172.21.15.94:0/3726724674 wait complete. 2024-05-27T12:35:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7f208d600640 1 -- 172.21.15.94:0/3017232484 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20900d4910 con 0x7f20880e43f0 2024-05-27T12:35:07.471 INFO:teuthology.orchestra.run.smithi094.stdout:55834574852 2024-05-27T12:35:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 msgr2=0x7f786806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 0x7f786806eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f787c036000 tx=0x7f787c030000 comp rx=0 tx=0).stop 2024-05-27T12:35:07.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 msgr2=0x7f78880b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.470+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f788000b970 tx=0x7f788000be30 comp rx=0 tx=0).stop 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 shutdown_connections 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f786806c9f0 0x7f786806eeb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f788815af50 0x7f78880b3f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78880b4fb0 0x7f7888155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 --2- 172.21.15.94:0/2096359153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f788806b930 0x7f788814de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 >> 172.21.15.94:0/2096359153 conn(0x7f78880b2660 msgr2=0x7f78880b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 shutdown_connections 2024-05-27T12:35:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.471+0000 7f7890200640 1 -- 172.21.15.94:0/2096359153 wait complete. 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 msgr2=0x7f207006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 0x7f207006ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2084009b60 tx=0x7f2084009190 comp rx=0 tx=0).stop 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 msgr2=0x7f20880e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 0x7f20880e3250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2090050200 tx=0x7f20900694e0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 shutdown_connections 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f207006c9c0 0x7f207006ee80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20880e43f0 0x7f20880e3250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2088038da0 0x7f20880339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 --2- 172.21.15.94:0/3017232484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20880389c0 0x7f208804edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.474+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 >> 172.21.15.94:0/3017232484 conn(0x7f2088053750 msgr2=0x7f208804d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.475+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 shutdown_connections 2024-05-27T12:35:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.475+0000 7f2096c00640 1 -- 172.21.15.94:0/3017232484 wait complete. 2024-05-27T12:35:07.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.477+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f58300ba040 con 0x7f58140051c0 2024-05-27T12:35:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.478+0000 7f5827e00640 1 -- 172.21.15.94:0/2986089377 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f582805a4d0 con 0x7f58140051c0 2024-05-27T12:35:07.477 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-05-27T12:35:07.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-05-27T12:35:07.478 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.481+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 msgr2=0x7f580c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.481+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 0x7f580c06ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5830144af0 tx=0x7f5820009040 comp rx=0 tx=0).stop 2024-05-27T12:35:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.482+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 msgr2=0x7f5830153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.482+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5828009fb0 tx=0x7f582800f580 comp rx=0 tx=0).stop 2024-05-27T12:35:07.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-27T12:35:07.484 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 shutdown_connections 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f580c06c940 0x7f580c06ee00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58300ccc90 0x7f58300cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58300c8390 0x7f58300c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 --2- 172.21.15.94:0/2986089377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140051c0 0x7f5830153920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 >> 172.21.15.94:0/2986089377 conn(0x7f5830067a50 msgr2=0x7f58300c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 shutdown_connections 2024-05-27T12:35:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.484+0000 7f5836c00640 1 -- 172.21.15.94:0/2986089377 wait complete. 2024-05-27T12:35:07.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-05-27T12:35:07.490 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-05-27T12:35:07.494 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-05-27T12:35:07.495 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-05-27T12:35:07.498 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-05-27T12:35:07.503 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:07.503 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-27T12:35:07.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:35:07.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.725+0000 7f2f5be00640 1 Processor -- start 2024-05-27T12:35:07.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.725+0000 7f2f5be00640 1 -- start start 2024-05-27T12:35:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.726+0000 7f2f5be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f5c13e600 0x7f2f5c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.726+0000 7f2f5be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.726+0000 7f2f5be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5c13e9e0 con 0x7f2f5c0cd480 2024-05-27T12:35:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.726+0000 7f2f5be00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5c15ba50 con 0x7f2f5c13e600 2024-05-27T12:35:07.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.726+0000 7f2f5be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5c15bd10 con 0x7f2f5c06eed0 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f5c13e600 0x7f2f5c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f5c13e600 0x7f2f5c071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56692/0 (socket says 172.21.15.94:56692) 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 -- 172.21.15.94:0/2602756389 learned_addr learned my addr 172.21.15.94:0/2602756389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5a000640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 -- 172.21.15.94:0/2602756389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 msgr2=0x7f2f5c06f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c06f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 -- 172.21.15.94:0/2602756389 >> v1:172.21.15.97:6789/0 conn(0x7f2f5c0cd480 legacy=0x7f2f5c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:07.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.727+0000 7f2f5aa00640 1 -- 172.21.15.94:0/2602756389 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c15bff0 con 0x7f2f5c13e600 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.728+0000 7f2f5aa00640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c071d60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2800721bca85bac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.728+0000 7f2f59600640 1 -- 172.21.15.94:0/2602756389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c036070 con 0x7f2f5c13e600 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.728+0000 7f2f59600640 1 -- 172.21.15.94:0/2602756389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c031600 con 0x7f2f5c13e600 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.728+0000 7f2f59600640 1 -- 172.21.15.94:0/2602756389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0319d0 con 0x7f2f5c13e600 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 -- 172.21.15.94:0/2602756389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 msgr2=0x7f2f5c071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c071d60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c02a6c0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 -- 172.21.15.94:0/2602756389 shutdown_connections 2024-05-27T12:35:07.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 --2- 172.21.15.94:0/2602756389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c071d60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 -- 172.21.15.94:0/2602756389 >> 172.21.15.94:0/2602756389 conn(0x7f2f5c0b2160 msgr2=0x7f2f5c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 -- 172.21.15.94:0/2602756389 shutdown_connections 2024-05-27T12:35:07.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.729+0000 7f2f5be00640 1 -- 172.21.15.94:0/2602756389 wait complete. 2024-05-27T12:35:07.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.731+0000 7f2f5be00640 1 Processor -- start 2024-05-27T12:35:07.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.731+0000 7f2f5be00640 1 -- start start 2024-05-27T12:35:07.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.732+0000 7f2f5be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c13fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f5c0cd480 0x7f2f5c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c15c920 con 0x7f2f5c13e600 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c15c5b0 con 0x7f2f5c0cd480 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5c141030 con 0x7f2f5c06eed0 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56700/0 (socket says 172.21.15.94:56700) 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 -- 172.21.15.94:0/1275321168 learned_addr learned my addr 172.21.15.94:0/1275321168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5aa00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c13fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5a000640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f5c0cd480 0x7f2f5c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 -- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 msgr2=0x7f2f5c13fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c13fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 -- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f5c0cd480 msgr2=0x7f2f5c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f5c0cd480 0x7f2f5c155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.733+0000 7f2f5b400640 1 -- 172.21.15.94:0/1275321168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c156e50 con 0x7f2f5c13e600 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f5b400640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c410 tx=0x7f2f5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011790 con 0x7f2f5c13e600 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54017070 con 0x7f2f5c13e600 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011b70 con 0x7f2f5c13e600 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c157130 con 0x7f2f5c13e600 2024-05-27T12:35:07.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.734+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c159690 con 0x7f2f5c13e600 2024-05-27T12:35:07.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.735+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5c075460 con 0x7f2f5c13e600 2024-05-27T12:35:07.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.736+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f54011d10 con 0x7f2f5c13e600 2024-05-27T12:35:07.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.737+0000 7f2f52a00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:07.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.737+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f5408b7b0 con 0x7f2f5c13e600 2024-05-27T12:35:07.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.737+0000 7f2f5aa00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:07.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.738+0000 7f2f5aa00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c033000 tx=0x7f2f4c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:07.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.741+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f54021020 con 0x7f2f5c13e600 2024-05-27T12:35:07.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.922+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f5c0b5260 con 0x7f2f3c06c8f0 2024-05-27T12:35:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.943+0000 7f2f52a00640 1 -- 172.21.15.94:0/1275321168 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f2f5c0b5260 con 0x7f2f3c06c8f0 2024-05-27T12:35:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:35:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 msgr2=0x7f2f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c033000 tx=0x7f2f4c009000 comp rx=0 tx=0).stop 2024-05-27T12:35:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 msgr2=0x7f2f5c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c410 tx=0x7f2f5400c8d0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 shutdown_connections 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f3c06c8f0 0x7f2f3c06edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f5c0cd480 0x7f2f5c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f5c13e600 0x7f2f5c156380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 --2- 172.21.15.94:0/1275321168 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f5c06eed0 0x7f2f5c13fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 >> 172.21.15.94:0/1275321168 conn(0x7f2f5c0b2160 msgr2=0x7f2f5c0adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 shutdown_connections 2024-05-27T12:35:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:07.949+0000 7f2f5be00640 1 -- 172.21.15.94:0/1275321168 wait complete. 2024-05-27T12:35:07.967 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-27T12:35:06.860794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.978481"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982878+0000","last_change":"2024-05-27T12:34:59.958231+0000","last_active":"2024-05-27T12:35:00.982878+0000","last_peered":"2024-05-27T12:35:00.982878+0000","last_clean":"2024-05-27T12:35:00.982878+0000","last_became_active":"2024-05-27T12:34:59.957423+0000","last_became_peered":"2024-05-27T12:34:59.957423+0000","last_unstale":"2024-05-27T12:35:00.982878+0000","last_undegraded":"2024-05-27T12:35:00.982878+0000","last_fullsized":"2024-05-27T12:35:00.982878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944746+0000","last_change":"2024-05-27T12:34:59.945994+0000","last_active":"2024-05-27T12:35:00.944746+0000","last_peered":"2024-05-27T12:35:00.944746+0000","last_clean":"2024-05-27T12:35:00.944746+0000","last_became_active":"2024-05-27T12:34:59.945737+0000","last_became_peered":"2024-05-27T12:34:59.945737+0000","last_unstale":"2024-05-27T12:35:00.944746+0000","last_undegraded":"2024-05-27T12:35:00.944746+0000","last_fullsized":"2024-05-27T12:35:00.944746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983788+0000","last_change":"2024-05-27T12:34:59.941922+0000","last_active":"2024-05-27T12:35:00.983788+0000","last_peered":"2024-05-27T12:35:00.983788+0000","last_clean":"2024-05-27T12:35:00.983788+0000","last_became_active":"2024-05-27T12:34:59.941543+0000","last_became_peered":"2024-05-27T12:34:59.941543+0000","last_unstale":"2024-05-27T12:35:00.983788+0000","last_undegraded":"2024-05-27T12:35:00.983788+0000","last_fullsized":"2024-05-27T12:35:00.983788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979987+0000","last_change":"2024-05-27T12:34:59.946464+0000","last_active":"2024-05-27T12:35:00.979987+0000","last_peered":"2024-05-27T12:35:00.979987+0000","last_clean":"2024-05-27T12:35:00.979987+0000","last_became_active":"2024-05-27T12:34:59.943307+0000","last_became_peered":"2024-05-27T12:34:59.943307+0000","last_unstale":"2024-05-27T12:35:00.979987+0000","last_undegraded":"2024-05-27T12:35:00.979987+0000","last_fullsized":"2024-05-27T12:35:00.979987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982559+0000","last_change":"2024-05-27T12:34:59.953536+0000","last_active":"2024-05-27T12:35:00.982559+0000","last_peered":"2024-05-27T12:35:00.982559+0000","last_clean":"2024-05-27T12:35:00.982559+0000","last_became_active":"2024-05-27T12:34:59.953266+0000","last_became_peered":"2024-05-27T12:34:59.953266+0000","last_unstale":"2024-05-27T12:35:00.982559+0000","last_undegraded":"2024-05-27T12:35:00.982559+0000","last_fullsized":"2024-05-27T12:35:00.982559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979484+0000","last_change":"2024-05-27T12:34:59.939835+0000","last_active":"2024-05-27T12:35:00.979484+0000","last_peered":"2024-05-27T12:35:00.979484+0000","last_clean":"2024-05-27T12:35:00.979484+0000","last_became_active":"2024-05-27T12:34:59.938363+0000","last_became_peered":"2024-05-27T12:34:59.938363+0000","last_unstale":"2024-05-27T12:35:00.979484+0000","last_undegraded":"2024-05-27T12:35:00.979484+0000","last_fullsized":"2024-05-27T12:35:00.979484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981481+0000","last_change":"2024-05-27T12:34:59.949943+0000","last_active":"2024-05-27T12:35:00.981481+0000","last_peered":"2024-05-27T12:35:00.981481+0000","last_clean":"2024-05-27T12:35:00.981481+0000","last_became_active":"2024-05-27T12:34:59.949473+0000","last_became_peered":"2024-05-27T12:34:59.949473+0000","last_unstale":"2024-05-27T12:35:00.981481+0000","last_undegraded":"2024-05-27T12:35:00.981481+0000","last_fullsized":"2024-05-27T12:35:00.981481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980190+0000","last_change":"2024-05-27T12:34:59.954206+0000","last_active":"2024-05-27T12:35:00.980190+0000","last_peered":"2024-05-27T12:35:00.980190+0000","last_clean":"2024-05-27T12:35:00.980190+0000","last_became_active":"2024-05-27T12:34:59.953560+0000","last_became_peered":"2024-05-27T12:34:59.953560+0000","last_unstale":"2024-05-27T12:35:00.980190+0000","last_undegraded":"2024-05-27T12:35:00.980190+0000","last_fullsized":"2024-05-27T12:35:00.980190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982883+0000","last_change":"2024-05-27T12:34:59.960424+0000","last_active":"2024-05-27T12:35:00.982883+0000","last_peered":"2024-05-27T12:35:00.982883+0000","last_clean":"2024-05-27T12:35:00.982883+0000","last_became_active":"2024-05-27T12:34:59.959909+0000","last_became_peered":"2024-05-27T12:34:59.959909+0000","last_unstale":"2024-05-27T12:35:00.982883+0000","last_undegraded":"2024-05-27T12:35:00.982883+0000","last_fullsized":"2024-05-27T12:35:00.982883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945456+0000","last_change":"2024-05-27T12:34:59.953459+0000","last_active":"2024-05-27T12:35:00.945456+0000","last_peered":"2024-05-27T12:35:00.945456+0000","last_clean":"2024-05-27T12:35:00.945456+0000","last_became_active":"2024-05-27T12:34:59.947619+0000","last_became_peered":"2024-05-27T12:34:59.947619+0000","last_unstale":"2024-05-27T12:35:00.945456+0000","last_undegraded":"2024-05-27T12:35:00.945456+0000","last_fullsized":"2024-05-27T12:35:00.945456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945452+0000","last_change":"2024-05-27T12:34:59.956011+0000","last_active":"2024-05-27T12:35:00.945452+0000","last_peered":"2024-05-27T12:35:00.945452+0000","last_clean":"2024-05-27T12:35:00.945452+0000","last_became_active":"2024-05-27T12:34:59.955642+0000","last_became_peered":"2024-05-27T12:34:59.955642+0000","last_unstale":"2024-05-27T12:35:00.945452+0000","last_undegraded":"2024-05-27T12:35:00.945452+0000","last_fullsized":"2024-05-27T12:35:00.945452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947001+0000","last_change":"2024-05-27T12:34:59.948810+0000","last_active":"2024-05-27T12:35:00.947001+0000","last_peered":"2024-05-27T12:35:00.947001+0000","last_clean":"2024-05-27T12:35:00.947001+0000","last_became_active":"2024-05-27T12:34:59.948353+0000","last_became_peered":"2024-05-27T12:34:59.948353+0000","last_unstale":"2024-05-27T12:35:00.947001+0000","last_undegraded":"2024-05-27T12:35:00.947001+0000","last_fullsized":"2024-05-27T12:35:00.947001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:52.789382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981338+0000","last_change":"2024-05-27T12:34:59.944238+0000","last_active":"2024-05-27T12:35:00.981338+0000","last_peered":"2024-05-27T12:35:00.981338+0000","last_clean":"2024-05-27T12:35:00.981338+0000","last_became_active":"2024-05-27T12:34:59.943490+0000","last_became_peered":"2024-05-27T12:34:59.943490+0000","last_unstale":"2024-05-27T12:35:00.981338+0000","last_undegraded":"2024-05-27T12:35:00.981338+0000","last_fullsized":"2024-05-27T12:35:00.981338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983782+0000","last_change":"2024-05-27T12:34:59.956775+0000","last_active":"2024-05-27T12:35:00.983782+0000","last_peered":"2024-05-27T12:35:00.983782+0000","last_clean":"2024-05-27T12:35:00.983782+0000","last_became_active":"2024-05-27T12:34:59.954558+0000","last_became_peered":"2024-05-27T12:34:59.954558+0000","last_unstale":"2024-05-27T12:35:00.983782+0000","last_undegraded":"2024-05-27T12:35:00.983782+0000","last_fullsized":"2024-05-27T12:35:00.983782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944437+0000","last_change":"2024-05-27T12:34:59.952264+0000","last_active":"2024-05-27T12:35:00.944437+0000","last_peered":"2024-05-27T12:35:00.944437+0000","last_clean":"2024-05-27T12:35:00.944437+0000","last_became_active":"2024-05-27T12:34:59.951960+0000","last_became_peered":"2024-05-27T12:34:59.951960+0000","last_unstale":"2024-05-27T12:35:00.944437+0000","last_undegraded":"2024-05-27T12:35:00.944437+0000","last_fullsized":"2024-05-27T12:35:00.944437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979305+0000","last_change":"2024-05-27T12:34:59.936327+0000","last_active":"2024-05-27T12:35:00.979305+0000","last_peered":"2024-05-27T12:35:00.979305+0000","last_clean":"2024-05-27T12:35:00.979305+0000","last_became_active":"2024-05-27T12:34:59.935576+0000","last_became_peered":"2024-05-27T12:34:59.935576+0000","last_unstale":"2024-05-27T12:35:00.979305+0000","last_undegraded":"2024-05-27T12:35:00.979305+0000","last_fullsized":"2024-05-27T12:35:00.979305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980066+0000","last_change":"2024-05-27T12:34:59.956063+0000","last_active":"2024-05-27T12:35:00.980066+0000","last_peered":"2024-05-27T12:35:00.980066+0000","last_clean":"2024-05-27T12:35:00.980066+0000","last_became_active":"2024-05-27T12:34:59.955763+0000","last_became_peered":"2024-05-27T12:34:59.955763+0000","last_unstale":"2024-05-27T12:35:00.980066+0000","last_undegraded":"2024-05-27T12:35:00.980066+0000","last_fullsized":"2024-05-27T12:35:00.980066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981742+0000","last_change":"2024-05-27T12:34:59.937118+0000","last_active":"2024-05-27T12:35:00.981742+0000","last_peered":"2024-05-27T12:35:00.981742+0000","last_clean":"2024-05-27T12:35:00.981742+0000","last_became_active":"2024-05-27T12:34:59.936814+0000","last_became_peered":"2024-05-27T12:34:59.936814+0000","last_unstale":"2024-05-27T12:35:00.981742+0000","last_undegraded":"2024-05-27T12:35:00.981742+0000","last_fullsized":"2024-05-27T12:35:00.981742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:54.133638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983099+0000","last_change":"2024-05-27T12:34:59.952246+0000","last_active":"2024-05-27T12:35:00.983099+0000","last_peered":"2024-05-27T12:35:00.983099+0000","last_clean":"2024-05-27T12:35:00.983099+0000","last_became_active":"2024-05-27T12:34:59.951939+0000","last_became_peered":"2024-05-27T12:34:59.951939+0000","last_unstale":"2024-05-27T12:35:00.983099+0000","last_undegraded":"2024-05-27T12:35:00.983099+0000","last_fullsized":"2024-05-27T12:35:00.983099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945011+0000","last_change":"2024-05-27T12:34:59.947085+0000","last_active":"2024-05-27T12:35:00.945011+0000","last_peered":"2024-05-27T12:35:00.945011+0000","last_clean":"2024-05-27T12:35:00.945011+0000","last_became_active":"2024-05-27T12:34:59.946740+0000","last_became_peered":"2024-05-27T12:34:59.946740+0000","last_unstale":"2024-05-27T12:35:00.945011+0000","last_undegraded":"2024-05-27T12:35:00.945011+0000","last_fullsized":"2024-05-27T12:35:00.945011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982971+0000","last_change":"2024-05-27T12:34:59.956268+0000","last_active":"2024-05-27T12:35:00.982971+0000","last_peered":"2024-05-27T12:35:00.982971+0000","last_clean":"2024-05-27T12:35:00.982971+0000","last_became_active":"2024-05-27T12:34:59.950964+0000","last_became_peered":"2024-05-27T12:34:59.950964+0000","last_unstale":"2024-05-27T12:35:00.982971+0000","last_undegraded":"2024-05-27T12:35:00.982971+0000","last_fullsized":"2024-05-27T12:35:00.982971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:12.683859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979683+0000","last_change":"2024-05-27T12:34:59.956010+0000","last_active":"2024-05-27T12:35:00.979683+0000","last_peered":"2024-05-27T12:35:00.979683+0000","last_clean":"2024-05-27T12:35:00.979683+0000","last_became_active":"2024-05-27T12:34:59.955692+0000","last_became_peered":"2024-05-27T12:34:59.955692+0000","last_unstale":"2024-05-27T12:35:00.979683+0000","last_undegraded":"2024-05-27T12:35:00.979683+0000","last_fullsized":"2024-05-27T12:35:00.979683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980587+0000","last_change":"2024-05-27T12:34:59.950671+0000","last_active":"2024-05-27T12:35:00.980587+0000","last_peered":"2024-05-27T12:35:00.980587+0000","last_clean":"2024-05-27T12:35:00.980587+0000","last_became_active":"2024-05-27T12:34:59.950168+0000","last_became_peered":"2024-05-27T12:34:59.950168+0000","last_unstale":"2024-05-27T12:35:00.980587+0000","last_undegraded":"2024-05-27T12:35:00.980587+0000","last_fullsized":"2024-05-27T12:35:00.980587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979458+0000","last_change":"2024-05-27T12:34:59.947016+0000","last_active":"2024-05-27T12:35:00.979458+0000","last_peered":"2024-05-27T12:35:00.979458+0000","last_clean":"2024-05-27T12:35:00.979458+0000","last_became_active":"2024-05-27T12:34:59.946710+0000","last_became_peered":"2024-05-27T12:34:59.946710+0000","last_unstale":"2024-05-27T12:35:00.979458+0000","last_undegraded":"2024-05-27T12:35:00.979458+0000","last_fullsized":"2024-05-27T12:35:00.979458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982935+0000","last_change":"2024-05-27T12:34:59.952637+0000","last_active":"2024-05-27T12:35:00.982935+0000","last_peered":"2024-05-27T12:35:00.982935+0000","last_clean":"2024-05-27T12:35:00.982935+0000","last_became_active":"2024-05-27T12:34:59.951346+0000","last_became_peered":"2024-05-27T12:34:59.951346+0000","last_unstale":"2024-05-27T12:35:00.982935+0000","last_undegraded":"2024-05-27T12:35:00.982935+0000","last_fullsized":"2024-05-27T12:35:00.982935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945571+0000","last_change":"2024-05-27T12:34:59.958301+0000","last_active":"2024-05-27T12:35:00.945571+0000","last_peered":"2024-05-27T12:35:00.945571+0000","last_clean":"2024-05-27T12:35:00.945571+0000","last_became_active":"2024-05-27T12:34:59.957996+0000","last_became_peered":"2024-05-27T12:34:59.957996+0000","last_unstale":"2024-05-27T12:35:00.945571+0000","last_undegraded":"2024-05-27T12:35:00.945571+0000","last_fullsized":"2024-05-27T12:35:00.945571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944292+0000","last_change":"2024-05-27T12:34:59.949515+0000","last_active":"2024-05-27T12:35:00.944292+0000","last_peered":"2024-05-27T12:35:00.944292+0000","last_clean":"2024-05-27T12:35:00.944292+0000","last_became_active":"2024-05-27T12:34:59.945960+0000","last_became_peered":"2024-05-27T12:34:59.945960+0000","last_unstale":"2024-05-27T12:35:00.944292+0000","last_undegraded":"2024-05-27T12:35:00.944292+0000","last_fullsized":"2024-05-27T12:35:00.944292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980945+0000","last_change":"2024-05-27T12:34:59.950471+0000","last_active":"2024-05-27T12:35:00.980945+0000","last_peered":"2024-05-27T12:35:00.980945+0000","last_clean":"2024-05-27T12:35:00.980945+0000","last_became_active":"2024-05-27T12:34:59.950135+0000","last_became_peered":"2024-05-27T12:34:59.950135+0000","last_unstale":"2024-05-27T12:35:00.980945+0000","last_undegraded":"2024-05-27T12:35:00.980945+0000","last_fullsized":"2024-05-27T12:35:00.980945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983332+0000","last_change":"2024-05-27T12:34:59.942121+0000","last_active":"2024-05-27T12:35:00.983332+0000","last_peered":"2024-05-27T12:35:00.983332+0000","last_clean":"2024-05-27T12:35:00.983332+0000","last_became_active":"2024-05-27T12:34:59.941006+0000","last_became_peered":"2024-05-27T12:34:59.941006+0000","last_unstale":"2024-05-27T12:35:00.983332+0000","last_undegraded":"2024-05-27T12:35:00.983332+0000","last_fullsized":"2024-05-27T12:35:00.983332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979945+0000","last_change":"2024-05-27T12:34:59.957178+0000","last_active":"2024-05-27T12:35:00.979945+0000","last_peered":"2024-05-27T12:35:00.979945+0000","last_clean":"2024-05-27T12:35:00.979945+0000","last_became_active":"2024-05-27T12:34:59.956854+0000","last_became_peered":"2024-05-27T12:34:59.956854+0000","last_unstale":"2024-05-27T12:35:00.979945+0000","last_undegraded":"2024-05-27T12:35:00.979945+0000","last_fullsized":"2024-05-27T12:35:00.979945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980174+0000","last_change":"2024-05-27T12:34:59.954564+0000","last_active":"2024-05-27T12:35:00.980174+0000","last_peered":"2024-05-27T12:35:00.980174+0000","last_clean":"2024-05-27T12:35:00.980174+0000","last_became_active":"2024-05-27T12:34:59.953442+0000","last_became_peered":"2024-05-27T12:34:59.953442+0000","last_unstale":"2024-05-27T12:35:00.980174+0000","last_undegraded":"2024-05-27T12:35:00.980174+0000","last_fullsized":"2024-05-27T12:35:00.980174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946038+0000","last_change":"2024-05-27T12:34:59.936968+0000","last_active":"2024-05-27T12:35:00.946038+0000","last_peered":"2024-05-27T12:35:00.946038+0000","last_clean":"2024-05-27T12:35:00.946038+0000","last_became_active":"2024-05-27T12:34:59.935850+0000","last_became_peered":"2024-05-27T12:34:59.935850+0000","last_unstale":"2024-05-27T12:35:00.946038+0000","last_undegraded":"2024-05-27T12:35:00.946038+0000","last_fullsized":"2024-05-27T12:35:00.946038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947845+0000","last_change":"2024-05-27T12:34:59.955463+0000","last_active":"2024-05-27T12:35:00.947845+0000","last_peered":"2024-05-27T12:35:00.947845+0000","last_clean":"2024-05-27T12:35:00.947845+0000","last_became_active":"2024-05-27T12:34:59.952053+0000","last_became_peered":"2024-05-27T12:34:59.952053+0000","last_unstale":"2024-05-27T12:35:00.947845+0000","last_undegraded":"2024-05-27T12:35:00.947845+0000","last_fullsized":"2024-05-27T12:35:00.947845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981586+0000","last_change":"2024-05-27T12:34:58.926344+0000","last_active":"2024-05-27T12:35:00.981586+0000","last_peered":"2024-05-27T12:35:00.981586+0000","last_clean":"2024-05-27T12:35:00.981586+0000","last_became_active":"2024-05-27T12:34:58.926021+0000","last_became_peered":"2024-05-27T12:34:58.926021+0000","last_unstale":"2024-05-27T12:35:00.981586+0000","last_undegraded":"2024-05-27T12:35:00.981586+0000","last_fullsized":"2024-05-27T12:35:00.981586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:35:59.130525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983155+0000","last_change":"2024-05-27T12:34:59.948432+0000","last_active":"2024-05-27T12:35:00.983155+0000","last_peered":"2024-05-27T12:35:00.983155+0000","last_clean":"2024-05-27T12:35:00.983155+0000","last_became_active":"2024-05-27T12:34:59.946912+0000","last_became_peered":"2024-05-27T12:34:59.946912+0000","last_unstale":"2024-05-27T12:35:00.983155+0000","last_undegraded":"2024-05-27T12:35:00.983155+0000","last_fullsized":"2024-05-27T12:35:00.983155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945458+0000","last_change":"2024-05-27T12:34:58.918241+0000","last_active":"2024-05-27T12:35:00.945458+0000","last_peered":"2024-05-27T12:35:00.945458+0000","last_clean":"2024-05-27T12:35:00.945458+0000","last_became_active":"2024-05-27T12:34:58.917640+0000","last_became_peered":"2024-05-27T12:34:58.917640+0000","last_unstale":"2024-05-27T12:35:00.945458+0000","last_undegraded":"2024-05-27T12:35:00.945458+0000","last_fullsized":"2024-05-27T12:35:00.945458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980740+0000","last_change":"2024-05-27T12:34:58.912568+0000","last_active":"2024-05-27T12:35:00.980740+0000","last_peered":"2024-05-27T12:35:00.980740+0000","last_clean":"2024-05-27T12:35:00.980740+0000","last_became_active":"2024-05-27T12:34:58.912280+0000","last_became_peered":"2024-05-27T12:34:58.912280+0000","last_unstale":"2024-05-27T12:35:00.980740+0000","last_undegraded":"2024-05-27T12:35:00.980740+0000","last_fullsized":"2024-05-27T12:35:00.980740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980543+0000","last_change":"2024-05-27T12:34:59.947954+0000","last_active":"2024-05-27T12:35:00.980543+0000","last_peered":"2024-05-27T12:35:00.980543+0000","last_clean":"2024-05-27T12:35:00.980543+0000","last_became_active":"2024-05-27T12:34:59.947702+0000","last_became_peered":"2024-05-27T12:34:59.947702+0000","last_unstale":"2024-05-27T12:35:00.980543+0000","last_undegraded":"2024-05-27T12:35:00.980543+0000","last_fullsized":"2024-05-27T12:35:00.980543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979570+0000","last_change":"2024-05-27T12:34:58.918329+0000","last_active":"2024-05-27T12:35:00.979570+0000","last_peered":"2024-05-27T12:35:00.979570+0000","last_clean":"2024-05-27T12:35:00.979570+0000","last_became_active":"2024-05-27T12:34:58.918007+0000","last_became_peered":"2024-05-27T12:34:58.918007+0000","last_unstale":"2024-05-27T12:35:00.979570+0000","last_undegraded":"2024-05-27T12:35:00.979570+0000","last_fullsized":"2024-05-27T12:35:00.979570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943753+0000","last_change":"2024-05-27T12:34:59.952134+0000","last_active":"2024-05-27T12:35:00.943753+0000","last_peered":"2024-05-27T12:35:00.943753+0000","last_clean":"2024-05-27T12:35:00.943753+0000","last_became_active":"2024-05-27T12:34:59.951823+0000","last_became_peered":"2024-05-27T12:34:59.951823+0000","last_unstale":"2024-05-27T12:35:00.943753+0000","last_undegraded":"2024-05-27T12:35:00.943753+0000","last_fullsized":"2024-05-27T12:35:00.943753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945424+0000","last_change":"2024-05-27T12:34:58.925614+0000","last_active":"2024-05-27T12:35:00.945424+0000","last_peered":"2024-05-27T12:35:00.945424+0000","last_clean":"2024-05-27T12:35:00.945424+0000","last_became_active":"2024-05-27T12:34:58.925312+0000","last_became_peered":"2024-05-27T12:34:58.925312+0000","last_unstale":"2024-05-27T12:35:00.945424+0000","last_undegraded":"2024-05-27T12:35:00.945424+0000","last_fullsized":"2024-05-27T12:35:00.945424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979670+0000","last_change":"2024-05-27T12:34:59.958727+0000","last_active":"2024-05-27T12:35:00.979670+0000","last_peered":"2024-05-27T12:35:00.979670+0000","last_clean":"2024-05-27T12:35:00.979670+0000","last_became_active":"2024-05-27T12:34:59.958405+0000","last_became_peered":"2024-05-27T12:34:59.958405+0000","last_unstale":"2024-05-27T12:35:00.979670+0000","last_undegraded":"2024-05-27T12:35:00.979670+0000","last_fullsized":"2024-05-27T12:35:00.979670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979703+0000","last_change":"2024-05-27T12:34:58.914733+0000","last_active":"2024-05-27T12:35:00.979703+0000","last_peered":"2024-05-27T12:35:00.979703+0000","last_clean":"2024-05-27T12:35:00.979703+0000","last_became_active":"2024-05-27T12:34:58.914357+0000","last_became_peered":"2024-05-27T12:34:58.914357+0000","last_unstale":"2024-05-27T12:35:00.979703+0000","last_undegraded":"2024-05-27T12:35:00.979703+0000","last_fullsized":"2024-05-27T12:35:00.979703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984178+0000","last_change":"2024-05-27T12:34:59.947773+0000","last_active":"2024-05-27T12:35:00.984178+0000","last_peered":"2024-05-27T12:35:00.984178+0000","last_clean":"2024-05-27T12:35:00.984178+0000","last_became_active":"2024-05-27T12:34:59.946577+0000","last_became_peered":"2024-05-27T12:34:59.946577+0000","last_unstale":"2024-05-27T12:35:00.984178+0000","last_undegraded":"2024-05-27T12:35:00.984178+0000","last_fullsized":"2024-05-27T12:35:00.984178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944069+0000","last_change":"2024-05-27T12:34:58.926880+0000","last_active":"2024-05-27T12:35:00.944069+0000","last_peered":"2024-05-27T12:35:00.944069+0000","last_clean":"2024-05-27T12:35:00.944069+0000","last_became_active":"2024-05-27T12:34:58.926621+0000","last_became_peered":"2024-05-27T12:34:58.926621+0000","last_unstale":"2024-05-27T12:35:00.944069+0000","last_undegraded":"2024-05-27T12:35:00.944069+0000","last_fullsized":"2024-05-27T12:35:00.944069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947431+0000","last_change":"2024-05-27T12:34:59.949465+0000","last_active":"2024-05-27T12:35:00.947431+0000","last_peered":"2024-05-27T12:35:00.947431+0000","last_clean":"2024-05-27T12:35:00.947431+0000","last_became_active":"2024-05-27T12:34:59.949190+0000","last_became_peered":"2024-05-27T12:34:59.949190+0000","last_unstale":"2024-05-27T12:35:00.947431+0000","last_undegraded":"2024-05-27T12:35:00.947431+0000","last_fullsized":"2024-05-27T12:35:00.947431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:27.503111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945133+0000","last_change":"2024-05-27T12:34:58.905974+0000","last_active":"2024-05-27T12:35:00.945133+0000","last_peered":"2024-05-27T12:35:00.945133+0000","last_clean":"2024-05-27T12:35:00.945133+0000","last_became_active":"2024-05-27T12:34:58.905596+0000","last_became_peered":"2024-05-27T12:34:58.905596+0000","last_unstale":"2024-05-27T12:35:00.945133+0000","last_undegraded":"2024-05-27T12:35:00.945133+0000","last_fullsized":"2024-05-27T12:35:00.945133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982755+0000","last_change":"2024-05-27T12:34:59.953412+0000","last_active":"2024-05-27T12:35:00.982755+0000","last_peered":"2024-05-27T12:35:00.982755+0000","last_clean":"2024-05-27T12:35:00.982755+0000","last_became_active":"2024-05-27T12:34:59.953103+0000","last_became_peered":"2024-05-27T12:34:59.953103+0000","last_unstale":"2024-05-27T12:35:00.982755+0000","last_undegraded":"2024-05-27T12:35:00.982755+0000","last_fullsized":"2024-05-27T12:35:00.982755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983281+0000","last_change":"2024-05-27T12:34:58.909953+0000","last_active":"2024-05-27T12:35:00.983281+0000","last_peered":"2024-05-27T12:35:00.983281+0000","last_clean":"2024-05-27T12:35:00.983281+0000","last_became_active":"2024-05-27T12:34:58.909633+0000","last_became_peered":"2024-05-27T12:34:58.909633+0000","last_unstale":"2024-05-27T12:35:00.983281+0000","last_undegraded":"2024-05-27T12:35:00.983281+0000","last_fullsized":"2024-05-27T12:35:00.983281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:39.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983591+0000","last_change":"2024-05-27T12:34:59.955304+0000","last_active":"2024-05-27T12:35:00.983591+0000","last_peered":"2024-05-27T12:35:00.983591+0000","last_clean":"2024-05-27T12:35:00.983591+0000","last_became_active":"2024-05-27T12:34:59.954940+0000","last_became_peered":"2024-05-27T12:34:59.954940+0000","last_unstale":"2024-05-27T12:35:00.983591+0000","last_undegraded":"2024-05-27T12:35:00.983591+0000","last_fullsized":"2024-05-27T12:35:00.983591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980018+0000","last_change":"2024-05-27T12:34:58.914660+0000","last_active":"2024-05-27T12:35:00.980018+0000","last_peered":"2024-05-27T12:35:00.980018+0000","last_clean":"2024-05-27T12:35:00.980018+0000","last_became_active":"2024-05-27T12:34:58.914368+0000","last_became_peered":"2024-05-27T12:34:58.914368+0000","last_unstale":"2024-05-27T12:35:00.980018+0000","last_undegraded":"2024-05-27T12:35:00.980018+0000","last_fullsized":"2024-05-27T12:35:00.980018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982688+0000","last_change":"2024-05-27T12:34:59.951438+0000","last_active":"2024-05-27T12:35:00.982688+0000","last_peered":"2024-05-27T12:35:00.982688+0000","last_clean":"2024-05-27T12:35:00.982688+0000","last_became_active":"2024-05-27T12:34:59.951140+0000","last_became_peered":"2024-05-27T12:34:59.951140+0000","last_unstale":"2024-05-27T12:35:00.982688+0000","last_undegraded":"2024-05-27T12:35:00.982688+0000","last_fullsized":"2024-05-27T12:35:00.982688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944792+0000","last_change":"2024-05-27T12:34:58.918066+0000","last_active":"2024-05-27T12:35:00.944792+0000","last_peered":"2024-05-27T12:35:00.944792+0000","last_clean":"2024-05-27T12:35:00.944792+0000","last_became_active":"2024-05-27T12:34:58.917613+0000","last_became_peered":"2024-05-27T12:34:58.917613+0000","last_unstale":"2024-05-27T12:35:00.944792+0000","last_undegraded":"2024-05-27T12:35:00.944792+0000","last_fullsized":"2024-05-27T12:35:00.944792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944830+0000","last_change":"2024-05-27T12:34:59.943832+0000","last_active":"2024-05-27T12:35:00.944830+0000","last_peered":"2024-05-27T12:35:00.944830+0000","last_clean":"2024-05-27T12:35:00.944830+0000","last_became_active":"2024-05-27T12:34:59.943535+0000","last_became_peered":"2024-05-27T12:34:59.943535+0000","last_unstale":"2024-05-27T12:35:00.944830+0000","last_undegraded":"2024-05-27T12:35:00.944830+0000","last_fullsized":"2024-05-27T12:35:00.944830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979766+0000","last_change":"2024-05-27T12:34:58.897545+0000","last_active":"2024-05-27T12:35:00.979766+0000","last_peered":"2024-05-27T12:35:00.979766+0000","last_clean":"2024-05-27T12:35:00.979766+0000","last_became_active":"2024-05-27T12:34:58.895961+0000","last_became_peered":"2024-05-27T12:34:58.895961+0000","last_unstale":"2024-05-27T12:35:00.979766+0000","last_undegraded":"2024-05-27T12:35:00.979766+0000","last_fullsized":"2024-05-27T12:35:00.979766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981686+0000","last_change":"2024-05-27T12:34:59.947365+0000","last_active":"2024-05-27T12:35:00.981686+0000","last_peered":"2024-05-27T12:35:00.981686+0000","last_clean":"2024-05-27T12:35:00.981686+0000","last_became_active":"2024-05-27T12:34:59.947060+0000","last_became_peered":"2024-05-27T12:34:59.947060+0000","last_unstale":"2024-05-27T12:35:00.981686+0000","last_undegraded":"2024-05-27T12:35:00.981686+0000","last_fullsized":"2024-05-27T12:35:00.981686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946688+0000","last_change":"2024-05-27T12:34:58.910200+0000","last_active":"2024-05-27T12:35:00.946688+0000","last_peered":"2024-05-27T12:35:00.946688+0000","last_clean":"2024-05-27T12:35:00.946688+0000","last_became_active":"2024-05-27T12:34:58.909819+0000","last_became_peered":"2024-05-27T12:34:58.909819+0000","last_unstale":"2024-05-27T12:35:00.946688+0000","last_undegraded":"2024-05-27T12:35:00.946688+0000","last_fullsized":"2024-05-27T12:35:00.946688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983250+0000","last_change":"2024-05-27T12:34:59.960324+0000","last_active":"2024-05-27T12:35:00.983250+0000","last_peered":"2024-05-27T12:35:00.983250+0000","last_clean":"2024-05-27T12:35:00.983250+0000","last_became_active":"2024-05-27T12:34:59.958530+0000","last_became_peered":"2024-05-27T12:34:59.958530+0000","last_unstale":"2024-05-27T12:35:00.983250+0000","last_undegraded":"2024-05-27T12:35:00.983250+0000","last_fullsized":"2024-05-27T12:35:00.983250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:10:04.239389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982017+0000","last_change":"2024-05-27T12:34:58.905547+0000","last_active":"2024-05-27T12:35:00.982017+0000","last_peered":"2024-05-27T12:35:00.982017+0000","last_clean":"2024-05-27T12:35:00.982017+0000","last_became_active":"2024-05-27T12:34:58.904281+0000","last_became_peered":"2024-05-27T12:34:58.904281+0000","last_unstale":"2024-05-27T12:35:00.982017+0000","last_undegraded":"2024-05-27T12:35:00.982017+0000","last_fullsized":"2024-05-27T12:35:00.982017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983204+0000","last_change":"2024-05-27T12:34:59.960325+0000","last_active":"2024-05-27T12:35:00.983204+0000","last_peered":"2024-05-27T12:35:00.983204+0000","last_clean":"2024-05-27T12:35:00.983204+0000","last_became_active":"2024-05-27T12:34:59.959764+0000","last_became_peered":"2024-05-27T12:34:59.959764+0000","last_unstale":"2024-05-27T12:35:00.983204+0000","last_undegraded":"2024-05-27T12:35:00.983204+0000","last_fullsized":"2024-05-27T12:35:00.983204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:26:24.248099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944296+0000","last_change":"2024-05-27T12:34:58.916163+0000","last_active":"2024-05-27T12:35:00.944296+0000","last_peered":"2024-05-27T12:35:00.944296+0000","last_clean":"2024-05-27T12:35:00.944296+0000","last_became_active":"2024-05-27T12:34:58.915823+0000","last_became_peered":"2024-05-27T12:34:58.915823+0000","last_unstale":"2024-05-27T12:35:00.944296+0000","last_undegraded":"2024-05-27T12:35:00.944296+0000","last_fullsized":"2024-05-27T12:35:00.944296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980392+0000","last_change":"2024-05-27T12:34:59.944578+0000","last_active":"2024-05-27T12:35:00.980392+0000","last_peered":"2024-05-27T12:35:00.980392+0000","last_clean":"2024-05-27T12:35:00.980392+0000","last_became_active":"2024-05-27T12:34:59.941815+0000","last_became_peered":"2024-05-27T12:34:59.941815+0000","last_unstale":"2024-05-27T12:35:00.980392+0000","last_undegraded":"2024-05-27T12:35:00.980392+0000","last_fullsized":"2024-05-27T12:35:00.980392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980249+0000","last_change":"2024-05-27T12:34:58.921269+0000","last_active":"2024-05-27T12:35:00.980249+0000","last_peered":"2024-05-27T12:35:00.980249+0000","last_clean":"2024-05-27T12:35:00.980249+0000","last_became_active":"2024-05-27T12:34:58.920929+0000","last_became_peered":"2024-05-27T12:34:58.920929+0000","last_unstale":"2024-05-27T12:35:00.980249+0000","last_undegraded":"2024-05-27T12:35:00.980249+0000","last_fullsized":"2024-05-27T12:35:00.980249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982666+0000","last_change":"2024-05-27T12:34:59.959353+0000","last_active":"2024-05-27T12:35:00.982666+0000","last_peered":"2024-05-27T12:35:00.982666+0000","last_clean":"2024-05-27T12:35:00.982666+0000","last_became_active":"2024-05-27T12:34:59.959097+0000","last_became_peered":"2024-05-27T12:34:59.959097+0000","last_unstale":"2024-05-27T12:35:00.982666+0000","last_undegraded":"2024-05-27T12:35:00.982666+0000","last_fullsized":"2024-05-27T12:35:00.982666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981367+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:35:00.981367+0000","last_peered":"2024-05-27T12:35:00.981367+0000","last_clean":"2024-05-27T12:35:00.981367+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:35:00.981367+0000","last_undegraded":"2024-05-27T12:35:00.981367+0000","last_fullsized":"2024-05-27T12:35:00.981367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979291+0000","last_change":"2024-05-27T12:34:58.920919+0000","last_active":"2024-05-27T12:35:00.979291+0000","last_peered":"2024-05-27T12:35:00.979291+0000","last_clean":"2024-05-27T12:35:00.979291+0000","last_became_active":"2024-05-27T12:34:58.918391+0000","last_became_peered":"2024-05-27T12:34:58.918391+0000","last_unstale":"2024-05-27T12:35:00.979291+0000","last_undegraded":"2024-05-27T12:35:00.979291+0000","last_fullsized":"2024-05-27T12:35:00.979291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981108+0000","last_change":"2024-05-27T12:34:59.959595+0000","last_active":"2024-05-27T12:35:00.981108+0000","last_peered":"2024-05-27T12:35:00.981108+0000","last_clean":"2024-05-27T12:35:00.981108+0000","last_became_active":"2024-05-27T12:34:59.959298+0000","last_became_peered":"2024-05-27T12:34:59.959298+0000","last_unstale":"2024-05-27T12:35:00.981108+0000","last_undegraded":"2024-05-27T12:35:00.981108+0000","last_fullsized":"2024-05-27T12:35:00.981108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981337+0000","last_change":"2024-05-27T12:34:58.926438+0000","last_active":"2024-05-27T12:35:00.981337+0000","last_peered":"2024-05-27T12:35:00.981337+0000","last_clean":"2024-05-27T12:35:00.981337+0000","last_became_active":"2024-05-27T12:34:58.926205+0000","last_became_peered":"2024-05-27T12:34:58.926205+0000","last_unstale":"2024-05-27T12:35:00.981337+0000","last_undegraded":"2024-05-27T12:35:00.981337+0000","last_fullsized":"2024-05-27T12:35:00.981337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982896+0000","last_change":"2024-05-27T12:34:59.947056+0000","last_active":"2024-05-27T12:35:00.982896+0000","last_peered":"2024-05-27T12:35:00.982896+0000","last_clean":"2024-05-27T12:35:00.982896+0000","last_became_active":"2024-05-27T12:34:59.946724+0000","last_became_peered":"2024-05-27T12:34:59.946724+0000","last_unstale":"2024-05-27T12:35:00.982896+0000","last_undegraded":"2024-05-27T12:35:00.982896+0000","last_fullsized":"2024-05-27T12:35:00.982896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:04.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980419+0000","last_change":"2024-05-27T12:34:58.906765+0000","last_active":"2024-05-27T12:35:00.980419+0000","last_peered":"2024-05-27T12:35:00.980419+0000","last_clean":"2024-05-27T12:35:00.980419+0000","last_became_active":"2024-05-27T12:34:58.904674+0000","last_became_peered":"2024-05-27T12:34:58.904674+0000","last_unstale":"2024-05-27T12:35:00.980419+0000","last_undegraded":"2024-05-27T12:35:00.980419+0000","last_fullsized":"2024-05-27T12:35:00.980419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943526+0000","last_change":"2024-05-27T12:34:59.943677+0000","last_active":"2024-05-27T12:35:00.943526+0000","last_peered":"2024-05-27T12:35:00.943526+0000","last_clean":"2024-05-27T12:35:00.943526+0000","last_became_active":"2024-05-27T12:34:59.943293+0000","last_became_peered":"2024-05-27T12:34:59.943293+0000","last_unstale":"2024-05-27T12:35:00.943526+0000","last_undegraded":"2024-05-27T12:35:00.943526+0000","last_fullsized":"2024-05-27T12:35:00.943526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980228+0000","last_change":"2024-05-27T12:34:58.916105+0000","last_active":"2024-05-27T12:35:00.980228+0000","last_peered":"2024-05-27T12:35:00.980228+0000","last_clean":"2024-05-27T12:35:00.980228+0000","last_became_active":"2024-05-27T12:34:58.913880+0000","last_became_peered":"2024-05-27T12:34:58.913880+0000","last_unstale":"2024-05-27T12:35:00.980228+0000","last_undegraded":"2024-05-27T12:35:00.980228+0000","last_fullsized":"2024-05-27T12:35:00.980228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946999+0000","last_change":"2024-05-27T12:34:59.946006+0000","last_active":"2024-05-27T12:35:00.946999+0000","last_peered":"2024-05-27T12:35:00.946999+0000","last_clean":"2024-05-27T12:35:00.946999+0000","last_became_active":"2024-05-27T12:34:59.945730+0000","last_became_peered":"2024-05-27T12:34:59.945730+0000","last_unstale":"2024-05-27T12:35:00.946999+0000","last_undegraded":"2024-05-27T12:35:00.946999+0000","last_fullsized":"2024-05-27T12:35:00.946999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:24.764029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943841+0000","last_change":"2024-05-27T12:34:58.906953+0000","last_active":"2024-05-27T12:35:00.943841+0000","last_peered":"2024-05-27T12:35:00.943841+0000","last_clean":"2024-05-27T12:35:00.943841+0000","last_became_active":"2024-05-27T12:34:58.906672+0000","last_became_peered":"2024-05-27T12:34:58.906672+0000","last_unstale":"2024-05-27T12:35:00.943841+0000","last_undegraded":"2024-05-27T12:35:00.943841+0000","last_fullsized":"2024-05-27T12:35:00.943841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980531+0000","last_change":"2024-05-27T12:34:59.947648+0000","last_active":"2024-05-27T12:35:00.980531+0000","last_peered":"2024-05-27T12:35:00.980531+0000","last_clean":"2024-05-27T12:35:00.980531+0000","last_became_active":"2024-05-27T12:34:59.946323+0000","last_became_peered":"2024-05-27T12:34:59.946323+0000","last_unstale":"2024-05-27T12:35:00.980531+0000","last_undegraded":"2024-05-27T12:35:00.980531+0000","last_fullsized":"2024-05-27T12:35:00.980531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983010+0000","last_change":"2024-05-27T12:34:58.922207+0000","last_active":"2024-05-27T12:35:00.983010+0000","last_peered":"2024-05-27T12:35:00.983010+0000","last_clean":"2024-05-27T12:35:00.983010+0000","last_became_active":"2024-05-27T12:34:58.921862+0000","last_became_peered":"2024-05-27T12:34:58.921862+0000","last_unstale":"2024-05-27T12:35:00.983010+0000","last_undegraded":"2024-05-27T12:35:00.983010+0000","last_fullsized":"2024-05-27T12:35:00.983010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:44.514186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943666+0000","last_change":"2024-05-27T12:34:59.945417+0000","last_active":"2024-05-27T12:35:00.943666+0000","last_peered":"2024-05-27T12:35:00.943666+0000","last_clean":"2024-05-27T12:35:00.943666+0000","last_became_active":"2024-05-27T12:34:59.945068+0000","last_became_peered":"2024-05-27T12:34:59.945068+0000","last_unstale":"2024-05-27T12:35:00.943666+0000","last_undegraded":"2024-05-27T12:35:00.943666+0000","last_fullsized":"2024-05-27T12:35:00.943666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.965456+0000","last_change":"2024-05-27T12:34:58.914462+0000","last_active":"2024-05-27T12:35:00.965456+0000","last_peered":"2024-05-27T12:35:00.965456+0000","last_clean":"2024-05-27T12:35:00.965456+0000","last_became_active":"2024-05-27T12:34:58.914188+0000","last_became_peered":"2024-05-27T12:34:58.914188+0000","last_unstale":"2024-05-27T12:35:00.965456+0000","last_undegraded":"2024-05-27T12:35:00.965456+0000","last_fullsized":"2024-05-27T12:35:00.965456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983335+0000","last_change":"2024-05-27T12:34:59.952226+0000","last_active":"2024-05-27T12:35:00.983335+0000","last_peered":"2024-05-27T12:35:00.983335+0000","last_clean":"2024-05-27T12:35:00.983335+0000","last_became_active":"2024-05-27T12:34:59.951936+0000","last_became_peered":"2024-05-27T12:34:59.951936+0000","last_unstale":"2024-05-27T12:35:00.983335+0000","last_undegraded":"2024-05-27T12:35:00.983335+0000","last_fullsized":"2024-05-27T12:35:00.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945936+0000","last_change":"2024-05-27T12:34:58.919780+0000","last_active":"2024-05-27T12:35:00.945936+0000","last_peered":"2024-05-27T12:35:00.945936+0000","last_clean":"2024-05-27T12:35:00.945936+0000","last_became_active":"2024-05-27T12:34:58.919505+0000","last_became_peered":"2024-05-27T12:34:58.919505+0000","last_unstale":"2024-05-27T12:35:00.945936+0000","last_undegraded":"2024-05-27T12:35:00.945936+0000","last_fullsized":"2024-05-27T12:35:00.945936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984200+0000","last_change":"2024-05-27T12:34:59.946900+0000","last_active":"2024-05-27T12:35:00.984200+0000","last_peered":"2024-05-27T12:35:00.984200+0000","last_clean":"2024-05-27T12:35:00.984200+0000","last_became_active":"2024-05-27T12:34:59.945045+0000","last_became_peered":"2024-05-27T12:34:59.945045+0000","last_unstale":"2024-05-27T12:35:00.984200+0000","last_undegraded":"2024-05-27T12:35:00.984200+0000","last_fullsized":"2024-05-27T12:35:00.984200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981186+0000","last_change":"2024-05-27T12:34:58.920871+0000","last_active":"2024-05-27T12:35:00.981186+0000","last_peered":"2024-05-27T12:35:00.981186+0000","last_clean":"2024-05-27T12:35:00.981186+0000","last_became_active":"2024-05-27T12:34:58.917367+0000","last_became_peered":"2024-05-27T12:34:58.917367+0000","last_unstale":"2024-05-27T12:35:00.981186+0000","last_undegraded":"2024-05-27T12:35:00.981186+0000","last_fullsized":"2024-05-27T12:35:00.981186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946167+0000","last_change":"2024-05-27T12:34:59.939621+0000","last_active":"2024-05-27T12:35:00.946167+0000","last_peered":"2024-05-27T12:35:00.946167+0000","last_clean":"2024-05-27T12:35:00.946167+0000","last_became_active":"2024-05-27T12:34:59.939269+0000","last_became_peered":"2024-05-27T12:34:59.939269+0000","last_unstale":"2024-05-27T12:35:00.946167+0000","last_undegraded":"2024-05-27T12:35:00.946167+0000","last_fullsized":"2024-05-27T12:35:00.946167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:45.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944012+0000","last_change":"2024-05-27T12:34:58.922915+0000","last_active":"2024-05-27T12:35:00.944012+0000","last_peered":"2024-05-27T12:35:00.944012+0000","last_clean":"2024-05-27T12:35:00.944012+0000","last_became_active":"2024-05-27T12:34:58.922438+0000","last_became_peered":"2024-05-27T12:34:58.922438+0000","last_unstale":"2024-05-27T12:35:00.944012+0000","last_undegraded":"2024-05-27T12:35:00.944012+0000","last_fullsized":"2024-05-27T12:35:00.944012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980353+0000","last_change":"2024-05-27T12:34:59.947355+0000","last_active":"2024-05-27T12:35:00.980353+0000","last_peered":"2024-05-27T12:35:00.980353+0000","last_clean":"2024-05-27T12:35:00.980353+0000","last_became_active":"2024-05-27T12:34:59.947069+0000","last_became_peered":"2024-05-27T12:34:59.947069+0000","last_unstale":"2024-05-27T12:35:00.980353+0000","last_undegraded":"2024-05-27T12:35:00.980353+0000","last_fullsized":"2024-05-27T12:35:00.980353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943707+0000","last_change":"2024-05-27T12:34:58.911817+0000","last_active":"2024-05-27T12:35:00.943707+0000","last_peered":"2024-05-27T12:35:00.943707+0000","last_clean":"2024-05-27T12:35:00.943707+0000","last_became_active":"2024-05-27T12:34:58.911528+0000","last_became_peered":"2024-05-27T12:34:58.911528+0000","last_unstale":"2024-05-27T12:35:00.943707+0000","last_undegraded":"2024-05-27T12:35:00.943707+0000","last_fullsized":"2024-05-27T12:35:00.943707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981479+0000","last_change":"2024-05-27T12:34:59.944469+0000","last_active":"2024-05-27T12:35:00.981479+0000","last_peered":"2024-05-27T12:35:00.981479+0000","last_clean":"2024-05-27T12:35:00.981479+0000","last_became_active":"2024-05-27T12:34:59.943952+0000","last_became_peered":"2024-05-27T12:34:59.943952+0000","last_unstale":"2024-05-27T12:35:00.981479+0000","last_undegraded":"2024-05-27T12:35:00.981479+0000","last_fullsized":"2024-05-27T12:35:00.981479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982842+0000","last_change":"2024-05-27T12:34:58.928167+0000","last_active":"2024-05-27T12:35:00.982842+0000","last_peered":"2024-05-27T12:35:00.982842+0000","last_clean":"2024-05-27T12:35:00.982842+0000","last_became_active":"2024-05-27T12:34:58.927922+0000","last_became_peered":"2024-05-27T12:34:58.927922+0000","last_unstale":"2024-05-27T12:35:00.982842+0000","last_undegraded":"2024-05-27T12:35:00.982842+0000","last_fullsized":"2024-05-27T12:35:00.982842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:03.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945600+0000","last_change":"2024-05-27T12:34:59.951341+0000","last_active":"2024-05-27T12:35:00.945600+0000","last_peered":"2024-05-27T12:35:00.945600+0000","last_clean":"2024-05-27T12:35:00.945600+0000","last_became_active":"2024-05-27T12:34:59.949217+0000","last_became_peered":"2024-05-27T12:34:59.949217+0000","last_unstale":"2024-05-27T12:35:00.945600+0000","last_undegraded":"2024-05-27T12:35:00.945600+0000","last_fullsized":"2024-05-27T12:35:00.945600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980544+0000","last_change":"2024-05-27T12:34:59.954080+0000","last_active":"2024-05-27T12:35:00.980544+0000","last_peered":"2024-05-27T12:35:00.980544+0000","last_clean":"2024-05-27T12:35:00.980544+0000","last_became_active":"2024-05-27T12:34:59.946377+0000","last_became_peered":"2024-05-27T12:34:59.946377+0000","last_unstale":"2024-05-27T12:35:00.980544+0000","last_undegraded":"2024-05-27T12:35:00.980544+0000","last_fullsized":"2024-05-27T12:35:00.980544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982176+0000","last_change":"2024-05-27T12:34:58.926815+0000","last_active":"2024-05-27T12:35:00.982176+0000","last_peered":"2024-05-27T12:35:00.982176+0000","last_clean":"2024-05-27T12:35:00.982176+0000","last_became_active":"2024-05-27T12:34:58.926419+0000","last_became_peered":"2024-05-27T12:34:58.926419+0000","last_unstale":"2024-05-27T12:35:00.982176+0000","last_undegraded":"2024-05-27T12:35:00.982176+0000","last_fullsized":"2024-05-27T12:35:00.982176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:05.278200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980205+0000","last_change":"2024-05-27T12:34:59.942376+0000","last_active":"2024-05-27T12:35:00.980205+0000","last_peered":"2024-05-27T12:35:00.980205+0000","last_clean":"2024-05-27T12:35:00.980205+0000","last_became_active":"2024-05-27T12:34:59.941083+0000","last_became_peered":"2024-05-27T12:34:59.941083+0000","last_unstale":"2024-05-27T12:35:00.980205+0000","last_undegraded":"2024-05-27T12:35:00.980205+0000","last_fullsized":"2024-05-27T12:35:00.980205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982827+0000","last_change":"2024-05-27T12:34:58.927394+0000","last_active":"2024-05-27T12:35:00.982827+0000","last_peered":"2024-05-27T12:35:00.982827+0000","last_clean":"2024-05-27T12:35:00.982827+0000","last_became_active":"2024-05-27T12:34:58.927124+0000","last_became_peered":"2024-05-27T12:34:58.927124+0000","last_unstale":"2024-05-27T12:35:00.982827+0000","last_undegraded":"2024-05-27T12:35:00.982827+0000","last_fullsized":"2024-05-27T12:35:00.982827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:53.128806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984386+0000","last_change":"2024-05-27T12:34:59.946923+0000","last_active":"2024-05-27T12:35:00.984386+0000","last_peered":"2024-05-27T12:35:00.984386+0000","last_clean":"2024-05-27T12:35:00.984386+0000","last_became_active":"2024-05-27T12:34:59.946554+0000","last_became_peered":"2024-05-27T12:34:59.946554+0000","last_unstale":"2024-05-27T12:35:00.984386+0000","last_undegraded":"2024-05-27T12:35:00.984386+0000","last_fullsized":"2024-05-27T12:35:00.984386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981709+0000","last_change":"2024-05-27T12:34:58.912889+0000","last_active":"2024-05-27T12:35:00.981709+0000","last_peered":"2024-05-27T12:35:00.981709+0000","last_clean":"2024-05-27T12:35:00.981709+0000","last_became_active":"2024-05-27T12:34:58.912617+0000","last_became_peered":"2024-05-27T12:34:58.912617+0000","last_unstale":"2024-05-27T12:35:00.981709+0000","last_undegraded":"2024-05-27T12:35:00.981709+0000","last_fullsized":"2024-05-27T12:35:00.981709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981440+0000","last_change":"2024-05-27T12:34:59.941269+0000","last_active":"2024-05-27T12:35:00.981440+0000","last_peered":"2024-05-27T12:35:00.981440+0000","last_clean":"2024-05-27T12:35:00.981440+0000","last_became_active":"2024-05-27T12:34:59.941004+0000","last_became_peered":"2024-05-27T12:34:59.941004+0000","last_unstale":"2024-05-27T12:35:00.981440+0000","last_undegraded":"2024-05-27T12:35:00.981440+0000","last_fullsized":"2024-05-27T12:35:00.981440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984278+0000","last_change":"2024-05-27T12:34:58.918917+0000","last_active":"2024-05-27T12:35:00.984278+0000","last_peered":"2024-05-27T12:35:00.984278+0000","last_clean":"2024-05-27T12:35:00.984278+0000","last_became_active":"2024-05-27T12:34:58.918499+0000","last_became_peered":"2024-05-27T12:34:58.918499+0000","last_unstale":"2024-05-27T12:35:00.984278+0000","last_undegraded":"2024-05-27T12:35:00.984278+0000","last_fullsized":"2024-05-27T12:35:00.984278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:35:07.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:35:08.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.196+0000 7eff52200640 1 Processor -- start 2024-05-27T12:35:08.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.196+0000 7eff52200640 1 -- start start 2024-05-27T12:35:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff52200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 0x7eff4c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff52200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff52200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4c160360 con 0x7eff4c06f280 2024-05-27T12:35:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff52200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7eff4c160f70 con 0x7eff4c15a250 2024-05-27T12:35:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff52200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c160640 con 0x7eff4c0cfb10 2024-05-27T12:35:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff4b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 0x7eff4c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff4be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff4be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:58134/0 (socket says 172.21.15.94:58134) 2024-05-27T12:35:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.198+0000 7eff4be00640 1 -- 172.21.15.94:0/3914302350 learned_addr learned my addr 172.21.15.94:0/3914302350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.199+0000 7eff4be00640 1 -- 172.21.15.94:0/3914302350 >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 msgr2=0x7eff4c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.199+0000 7eff4be00640 1 --2- 172.21.15.94:0/3914302350 >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 0x7eff4c06ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.199+0000 7eff4be00640 1 -- 172.21.15.94:0/3914302350 >> v1:172.21.15.97:6789/0 conn(0x7eff4c06f280 legacy=0x7eff4c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.199+0000 7eff4be00640 1 -- 172.21.15.94:0/3914302350 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c160920 con 0x7eff4c0cfb10 2024-05-27T12:35:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.199+0000 7eff4b400640 1 --2- 172.21.15.94:0/3914302350 >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 0x7eff4c06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:35:08.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.200+0000 7eff4be00640 1 --2- 172.21.15.94:0/3914302350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff44009e50 tx=0x7eff4402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e3bb9232d2a487 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.201+0000 7eff4a000640 1 -- 172.21.15.94:0/3914302350 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44036070 con 0x7eff4c0cfb10 2024-05-27T12:35:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.201+0000 7eff4a000640 1 -- 172.21.15.94:0/3914302350 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff44031600 con 0x7eff4c0cfb10 2024-05-27T12:35:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.201+0000 7eff4a000640 1 -- 172.21.15.94:0/3914302350 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440319d0 con 0x7eff4c0cfb10 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 -- 172.21.15.94:0/3914302350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 msgr2=0x7eff4c0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 --2- 172.21.15.94:0/3914302350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff44009e50 tx=0x7eff4402a710 comp rx=0 tx=0).stop 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 -- 172.21.15.94:0/3914302350 shutdown_connections 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 --2- 172.21.15.94:0/3914302350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c0c5710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 --2- 172.21.15.94:0/3914302350 >> v2:172.21.15.97:3300/0 conn(0x7eff4c15a250 0x7eff4c06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 -- 172.21.15.94:0/3914302350 >> 172.21.15.94:0/3914302350 conn(0x7eff4c0b3830 msgr2=0x7eff4c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 -- 172.21.15.94:0/3914302350 shutdown_connections 2024-05-27T12:35:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.202+0000 7eff52200640 1 -- 172.21.15.94:0/3914302350 wait complete. 2024-05-27T12:35:08.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.204+0000 7eff52200640 1 Processor -- start 2024-05-27T12:35:08.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.204+0000 7eff52200640 1 -- start start 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.205+0000 7eff52200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06f280 0x7eff4c1f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.205+0000 7eff52200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff52200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff52200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c06e7f0 con 0x7eff4c15a250 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff52200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c15a630 con 0x7eff4c06f280 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff52200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c06e970 con 0x7eff4c0cfb10 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56704/0 (socket says 172.21.15.94:56704) 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 -- 172.21.15.94:0/3880846960 learned_addr learned my addr 172.21.15.94:0/3880846960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4b400640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06f280 0x7eff4c1f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4aa00640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 -- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 msgr2=0x7eff4c1f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c1f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 -- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06f280 msgr2=0x7eff4c1f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06f280 0x7eff4c1f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.206+0000 7eff4be00640 1 -- 172.21.15.94:0/3880846960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c15d0e0 con 0x7eff4c15a250 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff4be00640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f70 tx=0x7eff44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44041020 con 0x7eff4c15a250 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff44008730 con 0x7eff4c15a250 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44036040 con 0x7eff4c15a250 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c15d3c0 con 0x7eff4c15a250 2024-05-27T12:35:08.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.207+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c15d980 con 0x7eff4c15a250 2024-05-27T12:35:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.208+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c075460 con 0x7eff4c15a250 2024-05-27T12:35:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.209+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff44044020 con 0x7eff4c15a250 2024-05-27T12:35:08.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.210+0000 7eff43400640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 0x7eff2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.210+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff440ac650 con 0x7eff4c15a250 2024-05-27T12:35:08.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.210+0000 7eff4b400640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 0x7eff2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.212+0000 7eff4b400640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 0x7eff2c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009a30 tx=0x7eff3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.214+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff4404b030 con 0x7eff4c15a250 2024-05-27T12:35:08.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.396+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff4c06cf70 con 0x7eff2c06c8f0 2024-05-27T12:35:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.414+0000 7eff43400640 1 -- 172.21.15.94:0/3880846960 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7eff4c06cf70 con 0x7eff2c06c8f0 2024-05-27T12:35:08.417 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 msgr2=0x7eff2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 0x7eff2c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009a30 tx=0x7eff3c008040 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 msgr2=0x7eff4c1f24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f70 tx=0x7eff44004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 shutdown_connections 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06c8f0 0x7eff2c06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15a250 0x7eff4c1f24b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0cfb10 0x7eff4c1f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 --2- 172.21.15.94:0/3880846960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06f280 0x7eff4c1f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 >> 172.21.15.94:0/3880846960 conn(0x7eff4c0b3830 msgr2=0x7eff4c0b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 shutdown_connections 2024-05-27T12:35:08.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.419+0000 7eff52200640 1 -- 172.21.15.94:0/3880846960 wait complete. 2024-05-27T12:35:08.435 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-27T12:35:06.860794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.978481"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982878+0000","last_change":"2024-05-27T12:34:59.958231+0000","last_active":"2024-05-27T12:35:00.982878+0000","last_peered":"2024-05-27T12:35:00.982878+0000","last_clean":"2024-05-27T12:35:00.982878+0000","last_became_active":"2024-05-27T12:34:59.957423+0000","last_became_peered":"2024-05-27T12:34:59.957423+0000","last_unstale":"2024-05-27T12:35:00.982878+0000","last_undegraded":"2024-05-27T12:35:00.982878+0000","last_fullsized":"2024-05-27T12:35:00.982878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944746+0000","last_change":"2024-05-27T12:34:59.945994+0000","last_active":"2024-05-27T12:35:00.944746+0000","last_peered":"2024-05-27T12:35:00.944746+0000","last_clean":"2024-05-27T12:35:00.944746+0000","last_became_active":"2024-05-27T12:34:59.945737+0000","last_became_peered":"2024-05-27T12:34:59.945737+0000","last_unstale":"2024-05-27T12:35:00.944746+0000","last_undegraded":"2024-05-27T12:35:00.944746+0000","last_fullsized":"2024-05-27T12:35:00.944746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983788+0000","last_change":"2024-05-27T12:34:59.941922+0000","last_active":"2024-05-27T12:35:00.983788+0000","last_peered":"2024-05-27T12:35:00.983788+0000","last_clean":"2024-05-27T12:35:00.983788+0000","last_became_active":"2024-05-27T12:34:59.941543+0000","last_became_peered":"2024-05-27T12:34:59.941543+0000","last_unstale":"2024-05-27T12:35:00.983788+0000","last_undegraded":"2024-05-27T12:35:00.983788+0000","last_fullsized":"2024-05-27T12:35:00.983788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979987+0000","last_change":"2024-05-27T12:34:59.946464+0000","last_active":"2024-05-27T12:35:00.979987+0000","last_peered":"2024-05-27T12:35:00.979987+0000","last_clean":"2024-05-27T12:35:00.979987+0000","last_became_active":"2024-05-27T12:34:59.943307+0000","last_became_peered":"2024-05-27T12:34:59.943307+0000","last_unstale":"2024-05-27T12:35:00.979987+0000","last_undegraded":"2024-05-27T12:35:00.979987+0000","last_fullsized":"2024-05-27T12:35:00.979987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982559+0000","last_change":"2024-05-27T12:34:59.953536+0000","last_active":"2024-05-27T12:35:00.982559+0000","last_peered":"2024-05-27T12:35:00.982559+0000","last_clean":"2024-05-27T12:35:00.982559+0000","last_became_active":"2024-05-27T12:34:59.953266+0000","last_became_peered":"2024-05-27T12:34:59.953266+0000","last_unstale":"2024-05-27T12:35:00.982559+0000","last_undegraded":"2024-05-27T12:35:00.982559+0000","last_fullsized":"2024-05-27T12:35:00.982559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979484+0000","last_change":"2024-05-27T12:34:59.939835+0000","last_active":"2024-05-27T12:35:00.979484+0000","last_peered":"2024-05-27T12:35:00.979484+0000","last_clean":"2024-05-27T12:35:00.979484+0000","last_became_active":"2024-05-27T12:34:59.938363+0000","last_became_peered":"2024-05-27T12:34:59.938363+0000","last_unstale":"2024-05-27T12:35:00.979484+0000","last_undegraded":"2024-05-27T12:35:00.979484+0000","last_fullsized":"2024-05-27T12:35:00.979484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981481+0000","last_change":"2024-05-27T12:34:59.949943+0000","last_active":"2024-05-27T12:35:00.981481+0000","last_peered":"2024-05-27T12:35:00.981481+0000","last_clean":"2024-05-27T12:35:00.981481+0000","last_became_active":"2024-05-27T12:34:59.949473+0000","last_became_peered":"2024-05-27T12:34:59.949473+0000","last_unstale":"2024-05-27T12:35:00.981481+0000","last_undegraded":"2024-05-27T12:35:00.981481+0000","last_fullsized":"2024-05-27T12:35:00.981481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980190+0000","last_change":"2024-05-27T12:34:59.954206+0000","last_active":"2024-05-27T12:35:00.980190+0000","last_peered":"2024-05-27T12:35:00.980190+0000","last_clean":"2024-05-27T12:35:00.980190+0000","last_became_active":"2024-05-27T12:34:59.953560+0000","last_became_peered":"2024-05-27T12:34:59.953560+0000","last_unstale":"2024-05-27T12:35:00.980190+0000","last_undegraded":"2024-05-27T12:35:00.980190+0000","last_fullsized":"2024-05-27T12:35:00.980190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982883+0000","last_change":"2024-05-27T12:34:59.960424+0000","last_active":"2024-05-27T12:35:00.982883+0000","last_peered":"2024-05-27T12:35:00.982883+0000","last_clean":"2024-05-27T12:35:00.982883+0000","last_became_active":"2024-05-27T12:34:59.959909+0000","last_became_peered":"2024-05-27T12:34:59.959909+0000","last_unstale":"2024-05-27T12:35:00.982883+0000","last_undegraded":"2024-05-27T12:35:00.982883+0000","last_fullsized":"2024-05-27T12:35:00.982883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945456+0000","last_change":"2024-05-27T12:34:59.953459+0000","last_active":"2024-05-27T12:35:00.945456+0000","last_peered":"2024-05-27T12:35:00.945456+0000","last_clean":"2024-05-27T12:35:00.945456+0000","last_became_active":"2024-05-27T12:34:59.947619+0000","last_became_peered":"2024-05-27T12:34:59.947619+0000","last_unstale":"2024-05-27T12:35:00.945456+0000","last_undegraded":"2024-05-27T12:35:00.945456+0000","last_fullsized":"2024-05-27T12:35:00.945456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945452+0000","last_change":"2024-05-27T12:34:59.956011+0000","last_active":"2024-05-27T12:35:00.945452+0000","last_peered":"2024-05-27T12:35:00.945452+0000","last_clean":"2024-05-27T12:35:00.945452+0000","last_became_active":"2024-05-27T12:34:59.955642+0000","last_became_peered":"2024-05-27T12:34:59.955642+0000","last_unstale":"2024-05-27T12:35:00.945452+0000","last_undegraded":"2024-05-27T12:35:00.945452+0000","last_fullsized":"2024-05-27T12:35:00.945452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947001+0000","last_change":"2024-05-27T12:34:59.948810+0000","last_active":"2024-05-27T12:35:00.947001+0000","last_peered":"2024-05-27T12:35:00.947001+0000","last_clean":"2024-05-27T12:35:00.947001+0000","last_became_active":"2024-05-27T12:34:59.948353+0000","last_became_peered":"2024-05-27T12:34:59.948353+0000","last_unstale":"2024-05-27T12:35:00.947001+0000","last_undegraded":"2024-05-27T12:35:00.947001+0000","last_fullsized":"2024-05-27T12:35:00.947001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:52.789382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981338+0000","last_change":"2024-05-27T12:34:59.944238+0000","last_active":"2024-05-27T12:35:00.981338+0000","last_peered":"2024-05-27T12:35:00.981338+0000","last_clean":"2024-05-27T12:35:00.981338+0000","last_became_active":"2024-05-27T12:34:59.943490+0000","last_became_peered":"2024-05-27T12:34:59.943490+0000","last_unstale":"2024-05-27T12:35:00.981338+0000","last_undegraded":"2024-05-27T12:35:00.981338+0000","last_fullsized":"2024-05-27T12:35:00.981338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983782+0000","last_change":"2024-05-27T12:34:59.956775+0000","last_active":"2024-05-27T12:35:00.983782+0000","last_peered":"2024-05-27T12:35:00.983782+0000","last_clean":"2024-05-27T12:35:00.983782+0000","last_became_active":"2024-05-27T12:34:59.954558+0000","last_became_peered":"2024-05-27T12:34:59.954558+0000","last_unstale":"2024-05-27T12:35:00.983782+0000","last_undegraded":"2024-05-27T12:35:00.983782+0000","last_fullsized":"2024-05-27T12:35:00.983782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944437+0000","last_change":"2024-05-27T12:34:59.952264+0000","last_active":"2024-05-27T12:35:00.944437+0000","last_peered":"2024-05-27T12:35:00.944437+0000","last_clean":"2024-05-27T12:35:00.944437+0000","last_became_active":"2024-05-27T12:34:59.951960+0000","last_became_peered":"2024-05-27T12:34:59.951960+0000","last_unstale":"2024-05-27T12:35:00.944437+0000","last_undegraded":"2024-05-27T12:35:00.944437+0000","last_fullsized":"2024-05-27T12:35:00.944437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979305+0000","last_change":"2024-05-27T12:34:59.936327+0000","last_active":"2024-05-27T12:35:00.979305+0000","last_peered":"2024-05-27T12:35:00.979305+0000","last_clean":"2024-05-27T12:35:00.979305+0000","last_became_active":"2024-05-27T12:34:59.935576+0000","last_became_peered":"2024-05-27T12:34:59.935576+0000","last_unstale":"2024-05-27T12:35:00.979305+0000","last_undegraded":"2024-05-27T12:35:00.979305+0000","last_fullsized":"2024-05-27T12:35:00.979305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980066+0000","last_change":"2024-05-27T12:34:59.956063+0000","last_active":"2024-05-27T12:35:00.980066+0000","last_peered":"2024-05-27T12:35:00.980066+0000","last_clean":"2024-05-27T12:35:00.980066+0000","last_became_active":"2024-05-27T12:34:59.955763+0000","last_became_peered":"2024-05-27T12:34:59.955763+0000","last_unstale":"2024-05-27T12:35:00.980066+0000","last_undegraded":"2024-05-27T12:35:00.980066+0000","last_fullsized":"2024-05-27T12:35:00.980066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981742+0000","last_change":"2024-05-27T12:34:59.937118+0000","last_active":"2024-05-27T12:35:00.981742+0000","last_peered":"2024-05-27T12:35:00.981742+0000","last_clean":"2024-05-27T12:35:00.981742+0000","last_became_active":"2024-05-27T12:34:59.936814+0000","last_became_peered":"2024-05-27T12:34:59.936814+0000","last_unstale":"2024-05-27T12:35:00.981742+0000","last_undegraded":"2024-05-27T12:35:00.981742+0000","last_fullsized":"2024-05-27T12:35:00.981742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:54.133638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983099+0000","last_change":"2024-05-27T12:34:59.952246+0000","last_active":"2024-05-27T12:35:00.983099+0000","last_peered":"2024-05-27T12:35:00.983099+0000","last_clean":"2024-05-27T12:35:00.983099+0000","last_became_active":"2024-05-27T12:34:59.951939+0000","last_became_peered":"2024-05-27T12:34:59.951939+0000","last_unstale":"2024-05-27T12:35:00.983099+0000","last_undegraded":"2024-05-27T12:35:00.983099+0000","last_fullsized":"2024-05-27T12:35:00.983099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945011+0000","last_change":"2024-05-27T12:34:59.947085+0000","last_active":"2024-05-27T12:35:00.945011+0000","last_peered":"2024-05-27T12:35:00.945011+0000","last_clean":"2024-05-27T12:35:00.945011+0000","last_became_active":"2024-05-27T12:34:59.946740+0000","last_became_peered":"2024-05-27T12:34:59.946740+0000","last_unstale":"2024-05-27T12:35:00.945011+0000","last_undegraded":"2024-05-27T12:35:00.945011+0000","last_fullsized":"2024-05-27T12:35:00.945011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982971+0000","last_change":"2024-05-27T12:34:59.956268+0000","last_active":"2024-05-27T12:35:00.982971+0000","last_peered":"2024-05-27T12:35:00.982971+0000","last_clean":"2024-05-27T12:35:00.982971+0000","last_became_active":"2024-05-27T12:34:59.950964+0000","last_became_peered":"2024-05-27T12:34:59.950964+0000","last_unstale":"2024-05-27T12:35:00.982971+0000","last_undegraded":"2024-05-27T12:35:00.982971+0000","last_fullsized":"2024-05-27T12:35:00.982971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:12.683859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979683+0000","last_change":"2024-05-27T12:34:59.956010+0000","last_active":"2024-05-27T12:35:00.979683+0000","last_peered":"2024-05-27T12:35:00.979683+0000","last_clean":"2024-05-27T12:35:00.979683+0000","last_became_active":"2024-05-27T12:34:59.955692+0000","last_became_peered":"2024-05-27T12:34:59.955692+0000","last_unstale":"2024-05-27T12:35:00.979683+0000","last_undegraded":"2024-05-27T12:35:00.979683+0000","last_fullsized":"2024-05-27T12:35:00.979683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980587+0000","last_change":"2024-05-27T12:34:59.950671+0000","last_active":"2024-05-27T12:35:00.980587+0000","last_peered":"2024-05-27T12:35:00.980587+0000","last_clean":"2024-05-27T12:35:00.980587+0000","last_became_active":"2024-05-27T12:34:59.950168+0000","last_became_peered":"2024-05-27T12:34:59.950168+0000","last_unstale":"2024-05-27T12:35:00.980587+0000","last_undegraded":"2024-05-27T12:35:00.980587+0000","last_fullsized":"2024-05-27T12:35:00.980587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979458+0000","last_change":"2024-05-27T12:34:59.947016+0000","last_active":"2024-05-27T12:35:00.979458+0000","last_peered":"2024-05-27T12:35:00.979458+0000","last_clean":"2024-05-27T12:35:00.979458+0000","last_became_active":"2024-05-27T12:34:59.946710+0000","last_became_peered":"2024-05-27T12:34:59.946710+0000","last_unstale":"2024-05-27T12:35:00.979458+0000","last_undegraded":"2024-05-27T12:35:00.979458+0000","last_fullsized":"2024-05-27T12:35:00.979458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982935+0000","last_change":"2024-05-27T12:34:59.952637+0000","last_active":"2024-05-27T12:35:00.982935+0000","last_peered":"2024-05-27T12:35:00.982935+0000","last_clean":"2024-05-27T12:35:00.982935+0000","last_became_active":"2024-05-27T12:34:59.951346+0000","last_became_peered":"2024-05-27T12:34:59.951346+0000","last_unstale":"2024-05-27T12:35:00.982935+0000","last_undegraded":"2024-05-27T12:35:00.982935+0000","last_fullsized":"2024-05-27T12:35:00.982935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945571+0000","last_change":"2024-05-27T12:34:59.958301+0000","last_active":"2024-05-27T12:35:00.945571+0000","last_peered":"2024-05-27T12:35:00.945571+0000","last_clean":"2024-05-27T12:35:00.945571+0000","last_became_active":"2024-05-27T12:34:59.957996+0000","last_became_peered":"2024-05-27T12:34:59.957996+0000","last_unstale":"2024-05-27T12:35:00.945571+0000","last_undegraded":"2024-05-27T12:35:00.945571+0000","last_fullsized":"2024-05-27T12:35:00.945571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944292+0000","last_change":"2024-05-27T12:34:59.949515+0000","last_active":"2024-05-27T12:35:00.944292+0000","last_peered":"2024-05-27T12:35:00.944292+0000","last_clean":"2024-05-27T12:35:00.944292+0000","last_became_active":"2024-05-27T12:34:59.945960+0000","last_became_peered":"2024-05-27T12:34:59.945960+0000","last_unstale":"2024-05-27T12:35:00.944292+0000","last_undegraded":"2024-05-27T12:35:00.944292+0000","last_fullsized":"2024-05-27T12:35:00.944292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980945+0000","last_change":"2024-05-27T12:34:59.950471+0000","last_active":"2024-05-27T12:35:00.980945+0000","last_peered":"2024-05-27T12:35:00.980945+0000","last_clean":"2024-05-27T12:35:00.980945+0000","last_became_active":"2024-05-27T12:34:59.950135+0000","last_became_peered":"2024-05-27T12:34:59.950135+0000","last_unstale":"2024-05-27T12:35:00.980945+0000","last_undegraded":"2024-05-27T12:35:00.980945+0000","last_fullsized":"2024-05-27T12:35:00.980945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983332+0000","last_change":"2024-05-27T12:34:59.942121+0000","last_active":"2024-05-27T12:35:00.983332+0000","last_peered":"2024-05-27T12:35:00.983332+0000","last_clean":"2024-05-27T12:35:00.983332+0000","last_became_active":"2024-05-27T12:34:59.941006+0000","last_became_peered":"2024-05-27T12:34:59.941006+0000","last_unstale":"2024-05-27T12:35:00.983332+0000","last_undegraded":"2024-05-27T12:35:00.983332+0000","last_fullsized":"2024-05-27T12:35:00.983332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:46.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979945+0000","last_change":"2024-05-27T12:34:59.957178+0000","last_active":"2024-05-27T12:35:00.979945+0000","last_peered":"2024-05-27T12:35:00.979945+0000","last_clean":"2024-05-27T12:35:00.979945+0000","last_became_active":"2024-05-27T12:34:59.956854+0000","last_became_peered":"2024-05-27T12:34:59.956854+0000","last_unstale":"2024-05-27T12:35:00.979945+0000","last_undegraded":"2024-05-27T12:35:00.979945+0000","last_fullsized":"2024-05-27T12:35:00.979945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:16.788725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980174+0000","last_change":"2024-05-27T12:34:59.954564+0000","last_active":"2024-05-27T12:35:00.980174+0000","last_peered":"2024-05-27T12:35:00.980174+0000","last_clean":"2024-05-27T12:35:00.980174+0000","last_became_active":"2024-05-27T12:34:59.953442+0000","last_became_peered":"2024-05-27T12:34:59.953442+0000","last_unstale":"2024-05-27T12:35:00.980174+0000","last_undegraded":"2024-05-27T12:35:00.980174+0000","last_fullsized":"2024-05-27T12:35:00.980174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946038+0000","last_change":"2024-05-27T12:34:59.936968+0000","last_active":"2024-05-27T12:35:00.946038+0000","last_peered":"2024-05-27T12:35:00.946038+0000","last_clean":"2024-05-27T12:35:00.946038+0000","last_became_active":"2024-05-27T12:34:59.935850+0000","last_became_peered":"2024-05-27T12:34:59.935850+0000","last_unstale":"2024-05-27T12:35:00.946038+0000","last_undegraded":"2024-05-27T12:35:00.946038+0000","last_fullsized":"2024-05-27T12:35:00.946038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947845+0000","last_change":"2024-05-27T12:34:59.955463+0000","last_active":"2024-05-27T12:35:00.947845+0000","last_peered":"2024-05-27T12:35:00.947845+0000","last_clean":"2024-05-27T12:35:00.947845+0000","last_became_active":"2024-05-27T12:34:59.952053+0000","last_became_peered":"2024-05-27T12:34:59.952053+0000","last_unstale":"2024-05-27T12:35:00.947845+0000","last_undegraded":"2024-05-27T12:35:00.947845+0000","last_fullsized":"2024-05-27T12:35:00.947845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:06.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981586+0000","last_change":"2024-05-27T12:34:58.926344+0000","last_active":"2024-05-27T12:35:00.981586+0000","last_peered":"2024-05-27T12:35:00.981586+0000","last_clean":"2024-05-27T12:35:00.981586+0000","last_became_active":"2024-05-27T12:34:58.926021+0000","last_became_peered":"2024-05-27T12:34:58.926021+0000","last_unstale":"2024-05-27T12:35:00.981586+0000","last_undegraded":"2024-05-27T12:35:00.981586+0000","last_fullsized":"2024-05-27T12:35:00.981586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:35:59.130525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983155+0000","last_change":"2024-05-27T12:34:59.948432+0000","last_active":"2024-05-27T12:35:00.983155+0000","last_peered":"2024-05-27T12:35:00.983155+0000","last_clean":"2024-05-27T12:35:00.983155+0000","last_became_active":"2024-05-27T12:34:59.946912+0000","last_became_peered":"2024-05-27T12:34:59.946912+0000","last_unstale":"2024-05-27T12:35:00.983155+0000","last_undegraded":"2024-05-27T12:35:00.983155+0000","last_fullsized":"2024-05-27T12:35:00.983155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945458+0000","last_change":"2024-05-27T12:34:58.918241+0000","last_active":"2024-05-27T12:35:00.945458+0000","last_peered":"2024-05-27T12:35:00.945458+0000","last_clean":"2024-05-27T12:35:00.945458+0000","last_became_active":"2024-05-27T12:34:58.917640+0000","last_became_peered":"2024-05-27T12:34:58.917640+0000","last_unstale":"2024-05-27T12:35:00.945458+0000","last_undegraded":"2024-05-27T12:35:00.945458+0000","last_fullsized":"2024-05-27T12:35:00.945458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980740+0000","last_change":"2024-05-27T12:34:58.912568+0000","last_active":"2024-05-27T12:35:00.980740+0000","last_peered":"2024-05-27T12:35:00.980740+0000","last_clean":"2024-05-27T12:35:00.980740+0000","last_became_active":"2024-05-27T12:34:58.912280+0000","last_became_peered":"2024-05-27T12:34:58.912280+0000","last_unstale":"2024-05-27T12:35:00.980740+0000","last_undegraded":"2024-05-27T12:35:00.980740+0000","last_fullsized":"2024-05-27T12:35:00.980740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980543+0000","last_change":"2024-05-27T12:34:59.947954+0000","last_active":"2024-05-27T12:35:00.980543+0000","last_peered":"2024-05-27T12:35:00.980543+0000","last_clean":"2024-05-27T12:35:00.980543+0000","last_became_active":"2024-05-27T12:34:59.947702+0000","last_became_peered":"2024-05-27T12:34:59.947702+0000","last_unstale":"2024-05-27T12:35:00.980543+0000","last_undegraded":"2024-05-27T12:35:00.980543+0000","last_fullsized":"2024-05-27T12:35:00.980543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979570+0000","last_change":"2024-05-27T12:34:58.918329+0000","last_active":"2024-05-27T12:35:00.979570+0000","last_peered":"2024-05-27T12:35:00.979570+0000","last_clean":"2024-05-27T12:35:00.979570+0000","last_became_active":"2024-05-27T12:34:58.918007+0000","last_became_peered":"2024-05-27T12:34:58.918007+0000","last_unstale":"2024-05-27T12:35:00.979570+0000","last_undegraded":"2024-05-27T12:35:00.979570+0000","last_fullsized":"2024-05-27T12:35:00.979570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943753+0000","last_change":"2024-05-27T12:34:59.952134+0000","last_active":"2024-05-27T12:35:00.943753+0000","last_peered":"2024-05-27T12:35:00.943753+0000","last_clean":"2024-05-27T12:35:00.943753+0000","last_became_active":"2024-05-27T12:34:59.951823+0000","last_became_peered":"2024-05-27T12:34:59.951823+0000","last_unstale":"2024-05-27T12:35:00.943753+0000","last_undegraded":"2024-05-27T12:35:00.943753+0000","last_fullsized":"2024-05-27T12:35:00.943753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:06.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945424+0000","last_change":"2024-05-27T12:34:58.925614+0000","last_active":"2024-05-27T12:35:00.945424+0000","last_peered":"2024-05-27T12:35:00.945424+0000","last_clean":"2024-05-27T12:35:00.945424+0000","last_became_active":"2024-05-27T12:34:58.925312+0000","last_became_peered":"2024-05-27T12:34:58.925312+0000","last_unstale":"2024-05-27T12:35:00.945424+0000","last_undegraded":"2024-05-27T12:35:00.945424+0000","last_fullsized":"2024-05-27T12:35:00.945424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979670+0000","last_change":"2024-05-27T12:34:59.958727+0000","last_active":"2024-05-27T12:35:00.979670+0000","last_peered":"2024-05-27T12:35:00.979670+0000","last_clean":"2024-05-27T12:35:00.979670+0000","last_became_active":"2024-05-27T12:34:59.958405+0000","last_became_peered":"2024-05-27T12:34:59.958405+0000","last_unstale":"2024-05-27T12:35:00.979670+0000","last_undegraded":"2024-05-27T12:35:00.979670+0000","last_fullsized":"2024-05-27T12:35:00.979670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:15:11.109895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979703+0000","last_change":"2024-05-27T12:34:58.914733+0000","last_active":"2024-05-27T12:35:00.979703+0000","last_peered":"2024-05-27T12:35:00.979703+0000","last_clean":"2024-05-27T12:35:00.979703+0000","last_became_active":"2024-05-27T12:34:58.914357+0000","last_became_peered":"2024-05-27T12:34:58.914357+0000","last_unstale":"2024-05-27T12:35:00.979703+0000","last_undegraded":"2024-05-27T12:35:00.979703+0000","last_fullsized":"2024-05-27T12:35:00.979703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984178+0000","last_change":"2024-05-27T12:34:59.947773+0000","last_active":"2024-05-27T12:35:00.984178+0000","last_peered":"2024-05-27T12:35:00.984178+0000","last_clean":"2024-05-27T12:35:00.984178+0000","last_became_active":"2024-05-27T12:34:59.946577+0000","last_became_peered":"2024-05-27T12:34:59.946577+0000","last_unstale":"2024-05-27T12:35:00.984178+0000","last_undegraded":"2024-05-27T12:35:00.984178+0000","last_fullsized":"2024-05-27T12:35:00.984178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944069+0000","last_change":"2024-05-27T12:34:58.926880+0000","last_active":"2024-05-27T12:35:00.944069+0000","last_peered":"2024-05-27T12:35:00.944069+0000","last_clean":"2024-05-27T12:35:00.944069+0000","last_became_active":"2024-05-27T12:34:58.926621+0000","last_became_peered":"2024-05-27T12:34:58.926621+0000","last_unstale":"2024-05-27T12:35:00.944069+0000","last_undegraded":"2024-05-27T12:35:00.944069+0000","last_fullsized":"2024-05-27T12:35:00.944069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.947431+0000","last_change":"2024-05-27T12:34:59.949465+0000","last_active":"2024-05-27T12:35:00.947431+0000","last_peered":"2024-05-27T12:35:00.947431+0000","last_clean":"2024-05-27T12:35:00.947431+0000","last_became_active":"2024-05-27T12:34:59.949190+0000","last_became_peered":"2024-05-27T12:34:59.949190+0000","last_unstale":"2024-05-27T12:35:00.947431+0000","last_undegraded":"2024-05-27T12:35:00.947431+0000","last_fullsized":"2024-05-27T12:35:00.947431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:27.503111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945133+0000","last_change":"2024-05-27T12:34:58.905974+0000","last_active":"2024-05-27T12:35:00.945133+0000","last_peered":"2024-05-27T12:35:00.945133+0000","last_clean":"2024-05-27T12:35:00.945133+0000","last_became_active":"2024-05-27T12:34:58.905596+0000","last_became_peered":"2024-05-27T12:34:58.905596+0000","last_unstale":"2024-05-27T12:35:00.945133+0000","last_undegraded":"2024-05-27T12:35:00.945133+0000","last_fullsized":"2024-05-27T12:35:00.945133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982755+0000","last_change":"2024-05-27T12:34:59.953412+0000","last_active":"2024-05-27T12:35:00.982755+0000","last_peered":"2024-05-27T12:35:00.982755+0000","last_clean":"2024-05-27T12:35:00.982755+0000","last_became_active":"2024-05-27T12:34:59.953103+0000","last_became_peered":"2024-05-27T12:34:59.953103+0000","last_unstale":"2024-05-27T12:35:00.982755+0000","last_undegraded":"2024-05-27T12:35:00.982755+0000","last_fullsized":"2024-05-27T12:35:00.982755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983281+0000","last_change":"2024-05-27T12:34:58.909953+0000","last_active":"2024-05-27T12:35:00.983281+0000","last_peered":"2024-05-27T12:35:00.983281+0000","last_clean":"2024-05-27T12:35:00.983281+0000","last_became_active":"2024-05-27T12:34:58.909633+0000","last_became_peered":"2024-05-27T12:34:58.909633+0000","last_unstale":"2024-05-27T12:35:00.983281+0000","last_undegraded":"2024-05-27T12:35:00.983281+0000","last_fullsized":"2024-05-27T12:35:00.983281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:39.952297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983591+0000","last_change":"2024-05-27T12:34:59.955304+0000","last_active":"2024-05-27T12:35:00.983591+0000","last_peered":"2024-05-27T12:35:00.983591+0000","last_clean":"2024-05-27T12:35:00.983591+0000","last_became_active":"2024-05-27T12:34:59.954940+0000","last_became_peered":"2024-05-27T12:34:59.954940+0000","last_unstale":"2024-05-27T12:35:00.983591+0000","last_undegraded":"2024-05-27T12:35:00.983591+0000","last_fullsized":"2024-05-27T12:35:00.983591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980018+0000","last_change":"2024-05-27T12:34:58.914660+0000","last_active":"2024-05-27T12:35:00.980018+0000","last_peered":"2024-05-27T12:35:00.980018+0000","last_clean":"2024-05-27T12:35:00.980018+0000","last_became_active":"2024-05-27T12:34:58.914368+0000","last_became_peered":"2024-05-27T12:34:58.914368+0000","last_unstale":"2024-05-27T12:35:00.980018+0000","last_undegraded":"2024-05-27T12:35:00.980018+0000","last_fullsized":"2024-05-27T12:35:00.980018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982688+0000","last_change":"2024-05-27T12:34:59.951438+0000","last_active":"2024-05-27T12:35:00.982688+0000","last_peered":"2024-05-27T12:35:00.982688+0000","last_clean":"2024-05-27T12:35:00.982688+0000","last_became_active":"2024-05-27T12:34:59.951140+0000","last_became_peered":"2024-05-27T12:34:59.951140+0000","last_unstale":"2024-05-27T12:35:00.982688+0000","last_undegraded":"2024-05-27T12:35:00.982688+0000","last_fullsized":"2024-05-27T12:35:00.982688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:39:01.364791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944792+0000","last_change":"2024-05-27T12:34:58.918066+0000","last_active":"2024-05-27T12:35:00.944792+0000","last_peered":"2024-05-27T12:35:00.944792+0000","last_clean":"2024-05-27T12:35:00.944792+0000","last_became_active":"2024-05-27T12:34:58.917613+0000","last_became_peered":"2024-05-27T12:34:58.917613+0000","last_unstale":"2024-05-27T12:35:00.944792+0000","last_undegraded":"2024-05-27T12:35:00.944792+0000","last_fullsized":"2024-05-27T12:35:00.944792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944830+0000","last_change":"2024-05-27T12:34:59.943832+0000","last_active":"2024-05-27T12:35:00.944830+0000","last_peered":"2024-05-27T12:35:00.944830+0000","last_clean":"2024-05-27T12:35:00.944830+0000","last_became_active":"2024-05-27T12:34:59.943535+0000","last_became_peered":"2024-05-27T12:34:59.943535+0000","last_unstale":"2024-05-27T12:35:00.944830+0000","last_undegraded":"2024-05-27T12:35:00.944830+0000","last_fullsized":"2024-05-27T12:35:00.944830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:13:44.741714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979766+0000","last_change":"2024-05-27T12:34:58.897545+0000","last_active":"2024-05-27T12:35:00.979766+0000","last_peered":"2024-05-27T12:35:00.979766+0000","last_clean":"2024-05-27T12:35:00.979766+0000","last_became_active":"2024-05-27T12:34:58.895961+0000","last_became_peered":"2024-05-27T12:34:58.895961+0000","last_unstale":"2024-05-27T12:35:00.979766+0000","last_undegraded":"2024-05-27T12:35:00.979766+0000","last_fullsized":"2024-05-27T12:35:00.979766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981686+0000","last_change":"2024-05-27T12:34:59.947365+0000","last_active":"2024-05-27T12:35:00.981686+0000","last_peered":"2024-05-27T12:35:00.981686+0000","last_clean":"2024-05-27T12:35:00.981686+0000","last_became_active":"2024-05-27T12:34:59.947060+0000","last_became_peered":"2024-05-27T12:34:59.947060+0000","last_unstale":"2024-05-27T12:35:00.981686+0000","last_undegraded":"2024-05-27T12:35:00.981686+0000","last_fullsized":"2024-05-27T12:35:00.981686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946688+0000","last_change":"2024-05-27T12:34:58.910200+0000","last_active":"2024-05-27T12:35:00.946688+0000","last_peered":"2024-05-27T12:35:00.946688+0000","last_clean":"2024-05-27T12:35:00.946688+0000","last_became_active":"2024-05-27T12:34:58.909819+0000","last_became_peered":"2024-05-27T12:34:58.909819+0000","last_unstale":"2024-05-27T12:35:00.946688+0000","last_undegraded":"2024-05-27T12:35:00.946688+0000","last_fullsized":"2024-05-27T12:35:00.946688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983250+0000","last_change":"2024-05-27T12:34:59.960324+0000","last_active":"2024-05-27T12:35:00.983250+0000","last_peered":"2024-05-27T12:35:00.983250+0000","last_clean":"2024-05-27T12:35:00.983250+0000","last_became_active":"2024-05-27T12:34:59.958530+0000","last_became_peered":"2024-05-27T12:34:59.958530+0000","last_unstale":"2024-05-27T12:35:00.983250+0000","last_undegraded":"2024-05-27T12:35:00.983250+0000","last_fullsized":"2024-05-27T12:35:00.983250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:10:04.239389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982017+0000","last_change":"2024-05-27T12:34:58.905547+0000","last_active":"2024-05-27T12:35:00.982017+0000","last_peered":"2024-05-27T12:35:00.982017+0000","last_clean":"2024-05-27T12:35:00.982017+0000","last_became_active":"2024-05-27T12:34:58.904281+0000","last_became_peered":"2024-05-27T12:34:58.904281+0000","last_unstale":"2024-05-27T12:35:00.982017+0000","last_undegraded":"2024-05-27T12:35:00.982017+0000","last_fullsized":"2024-05-27T12:35:00.982017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983204+0000","last_change":"2024-05-27T12:34:59.960325+0000","last_active":"2024-05-27T12:35:00.983204+0000","last_peered":"2024-05-27T12:35:00.983204+0000","last_clean":"2024-05-27T12:35:00.983204+0000","last_became_active":"2024-05-27T12:34:59.959764+0000","last_became_peered":"2024-05-27T12:34:59.959764+0000","last_unstale":"2024-05-27T12:35:00.983204+0000","last_undegraded":"2024-05-27T12:35:00.983204+0000","last_fullsized":"2024-05-27T12:35:00.983204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:26:24.248099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944296+0000","last_change":"2024-05-27T12:34:58.916163+0000","last_active":"2024-05-27T12:35:00.944296+0000","last_peered":"2024-05-27T12:35:00.944296+0000","last_clean":"2024-05-27T12:35:00.944296+0000","last_became_active":"2024-05-27T12:34:58.915823+0000","last_became_peered":"2024-05-27T12:34:58.915823+0000","last_unstale":"2024-05-27T12:35:00.944296+0000","last_undegraded":"2024-05-27T12:35:00.944296+0000","last_fullsized":"2024-05-27T12:35:00.944296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980392+0000","last_change":"2024-05-27T12:34:59.944578+0000","last_active":"2024-05-27T12:35:00.980392+0000","last_peered":"2024-05-27T12:35:00.980392+0000","last_clean":"2024-05-27T12:35:00.980392+0000","last_became_active":"2024-05-27T12:34:59.941815+0000","last_became_peered":"2024-05-27T12:34:59.941815+0000","last_unstale":"2024-05-27T12:35:00.980392+0000","last_undegraded":"2024-05-27T12:35:00.980392+0000","last_fullsized":"2024-05-27T12:35:00.980392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:29:06.095633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980249+0000","last_change":"2024-05-27T12:34:58.921269+0000","last_active":"2024-05-27T12:35:00.980249+0000","last_peered":"2024-05-27T12:35:00.980249+0000","last_clean":"2024-05-27T12:35:00.980249+0000","last_became_active":"2024-05-27T12:34:58.920929+0000","last_became_peered":"2024-05-27T12:34:58.920929+0000","last_unstale":"2024-05-27T12:35:00.980249+0000","last_undegraded":"2024-05-27T12:35:00.980249+0000","last_fullsized":"2024-05-27T12:35:00.980249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982666+0000","last_change":"2024-05-27T12:34:59.959353+0000","last_active":"2024-05-27T12:35:00.982666+0000","last_peered":"2024-05-27T12:35:00.982666+0000","last_clean":"2024-05-27T12:35:00.982666+0000","last_became_active":"2024-05-27T12:34:59.959097+0000","last_became_peered":"2024-05-27T12:34:59.959097+0000","last_unstale":"2024-05-27T12:35:00.982666+0000","last_undegraded":"2024-05-27T12:35:00.982666+0000","last_fullsized":"2024-05-27T12:35:00.982666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:50.875875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981367+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:35:00.981367+0000","last_peered":"2024-05-27T12:35:00.981367+0000","last_clean":"2024-05-27T12:35:00.981367+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:35:00.981367+0000","last_undegraded":"2024-05-27T12:35:00.981367+0000","last_fullsized":"2024-05-27T12:35:00.981367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.979291+0000","last_change":"2024-05-27T12:34:58.920919+0000","last_active":"2024-05-27T12:35:00.979291+0000","last_peered":"2024-05-27T12:35:00.979291+0000","last_clean":"2024-05-27T12:35:00.979291+0000","last_became_active":"2024-05-27T12:34:58.918391+0000","last_became_peered":"2024-05-27T12:34:58.918391+0000","last_unstale":"2024-05-27T12:35:00.979291+0000","last_undegraded":"2024-05-27T12:35:00.979291+0000","last_fullsized":"2024-05-27T12:35:00.979291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981108+0000","last_change":"2024-05-27T12:34:59.959595+0000","last_active":"2024-05-27T12:35:00.981108+0000","last_peered":"2024-05-27T12:35:00.981108+0000","last_clean":"2024-05-27T12:35:00.981108+0000","last_became_active":"2024-05-27T12:34:59.959298+0000","last_became_peered":"2024-05-27T12:34:59.959298+0000","last_unstale":"2024-05-27T12:35:00.981108+0000","last_undegraded":"2024-05-27T12:35:00.981108+0000","last_fullsized":"2024-05-27T12:35:00.981108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981337+0000","last_change":"2024-05-27T12:34:58.926438+0000","last_active":"2024-05-27T12:35:00.981337+0000","last_peered":"2024-05-27T12:35:00.981337+0000","last_clean":"2024-05-27T12:35:00.981337+0000","last_became_active":"2024-05-27T12:34:58.926205+0000","last_became_peered":"2024-05-27T12:34:58.926205+0000","last_unstale":"2024-05-27T12:35:00.981337+0000","last_undegraded":"2024-05-27T12:35:00.981337+0000","last_fullsized":"2024-05-27T12:35:00.981337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982896+0000","last_change":"2024-05-27T12:34:59.947056+0000","last_active":"2024-05-27T12:35:00.982896+0000","last_peered":"2024-05-27T12:35:00.982896+0000","last_clean":"2024-05-27T12:35:00.982896+0000","last_became_active":"2024-05-27T12:34:59.946724+0000","last_became_peered":"2024-05-27T12:34:59.946724+0000","last_unstale":"2024-05-27T12:35:00.982896+0000","last_undegraded":"2024-05-27T12:35:00.982896+0000","last_fullsized":"2024-05-27T12:35:00.982896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:04.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980419+0000","last_change":"2024-05-27T12:34:58.906765+0000","last_active":"2024-05-27T12:35:00.980419+0000","last_peered":"2024-05-27T12:35:00.980419+0000","last_clean":"2024-05-27T12:35:00.980419+0000","last_became_active":"2024-05-27T12:34:58.904674+0000","last_became_peered":"2024-05-27T12:34:58.904674+0000","last_unstale":"2024-05-27T12:35:00.980419+0000","last_undegraded":"2024-05-27T12:35:00.980419+0000","last_fullsized":"2024-05-27T12:35:00.980419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:47:24.302171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943526+0000","last_change":"2024-05-27T12:34:59.943677+0000","last_active":"2024-05-27T12:35:00.943526+0000","last_peered":"2024-05-27T12:35:00.943526+0000","last_clean":"2024-05-27T12:35:00.943526+0000","last_became_active":"2024-05-27T12:34:59.943293+0000","last_became_peered":"2024-05-27T12:34:59.943293+0000","last_unstale":"2024-05-27T12:35:00.943526+0000","last_undegraded":"2024-05-27T12:35:00.943526+0000","last_fullsized":"2024-05-27T12:35:00.943526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980228+0000","last_change":"2024-05-27T12:34:58.916105+0000","last_active":"2024-05-27T12:35:00.980228+0000","last_peered":"2024-05-27T12:35:00.980228+0000","last_clean":"2024-05-27T12:35:00.980228+0000","last_became_active":"2024-05-27T12:34:58.913880+0000","last_became_peered":"2024-05-27T12:34:58.913880+0000","last_unstale":"2024-05-27T12:35:00.980228+0000","last_undegraded":"2024-05-27T12:35:00.980228+0000","last_fullsized":"2024-05-27T12:35:00.980228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946999+0000","last_change":"2024-05-27T12:34:59.946006+0000","last_active":"2024-05-27T12:35:00.946999+0000","last_peered":"2024-05-27T12:35:00.946999+0000","last_clean":"2024-05-27T12:35:00.946999+0000","last_became_active":"2024-05-27T12:34:59.945730+0000","last_became_peered":"2024-05-27T12:34:59.945730+0000","last_unstale":"2024-05-27T12:35:00.946999+0000","last_undegraded":"2024-05-27T12:35:00.946999+0000","last_fullsized":"2024-05-27T12:35:00.946999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:24.764029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943841+0000","last_change":"2024-05-27T12:34:58.906953+0000","last_active":"2024-05-27T12:35:00.943841+0000","last_peered":"2024-05-27T12:35:00.943841+0000","last_clean":"2024-05-27T12:35:00.943841+0000","last_became_active":"2024-05-27T12:34:58.906672+0000","last_became_peered":"2024-05-27T12:34:58.906672+0000","last_unstale":"2024-05-27T12:35:00.943841+0000","last_undegraded":"2024-05-27T12:35:00.943841+0000","last_fullsized":"2024-05-27T12:35:00.943841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:00:18.661384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980531+0000","last_change":"2024-05-27T12:34:59.947648+0000","last_active":"2024-05-27T12:35:00.980531+0000","last_peered":"2024-05-27T12:35:00.980531+0000","last_clean":"2024-05-27T12:35:00.980531+0000","last_became_active":"2024-05-27T12:34:59.946323+0000","last_became_peered":"2024-05-27T12:34:59.946323+0000","last_unstale":"2024-05-27T12:35:00.980531+0000","last_undegraded":"2024-05-27T12:35:00.980531+0000","last_fullsized":"2024-05-27T12:35:00.980531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:44:43.447547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983010+0000","last_change":"2024-05-27T12:34:58.922207+0000","last_active":"2024-05-27T12:35:00.983010+0000","last_peered":"2024-05-27T12:35:00.983010+0000","last_clean":"2024-05-27T12:35:00.983010+0000","last_became_active":"2024-05-27T12:34:58.921862+0000","last_became_peered":"2024-05-27T12:34:58.921862+0000","last_unstale":"2024-05-27T12:35:00.983010+0000","last_undegraded":"2024-05-27T12:35:00.983010+0000","last_fullsized":"2024-05-27T12:35:00.983010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:44.514186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943666+0000","last_change":"2024-05-27T12:34:59.945417+0000","last_active":"2024-05-27T12:35:00.943666+0000","last_peered":"2024-05-27T12:35:00.943666+0000","last_clean":"2024-05-27T12:35:00.943666+0000","last_became_active":"2024-05-27T12:34:59.945068+0000","last_became_peered":"2024-05-27T12:34:59.945068+0000","last_unstale":"2024-05-27T12:35:00.943666+0000","last_undegraded":"2024-05-27T12:35:00.943666+0000","last_fullsized":"2024-05-27T12:35:00.943666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:21:01.582242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.965456+0000","last_change":"2024-05-27T12:34:58.914462+0000","last_active":"2024-05-27T12:35:00.965456+0000","last_peered":"2024-05-27T12:35:00.965456+0000","last_clean":"2024-05-27T12:35:00.965456+0000","last_became_active":"2024-05-27T12:34:58.914188+0000","last_became_peered":"2024-05-27T12:34:58.914188+0000","last_unstale":"2024-05-27T12:35:00.965456+0000","last_undegraded":"2024-05-27T12:35:00.965456+0000","last_fullsized":"2024-05-27T12:35:00.965456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.983335+0000","last_change":"2024-05-27T12:34:59.952226+0000","last_active":"2024-05-27T12:35:00.983335+0000","last_peered":"2024-05-27T12:35:00.983335+0000","last_clean":"2024-05-27T12:35:00.983335+0000","last_became_active":"2024-05-27T12:34:59.951936+0000","last_became_peered":"2024-05-27T12:34:59.951936+0000","last_unstale":"2024-05-27T12:35:00.983335+0000","last_undegraded":"2024-05-27T12:35:00.983335+0000","last_fullsized":"2024-05-27T12:35:00.983335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:36:52.693332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945936+0000","last_change":"2024-05-27T12:34:58.919780+0000","last_active":"2024-05-27T12:35:00.945936+0000","last_peered":"2024-05-27T12:35:00.945936+0000","last_clean":"2024-05-27T12:35:00.945936+0000","last_became_active":"2024-05-27T12:34:58.919505+0000","last_became_peered":"2024-05-27T12:34:58.919505+0000","last_unstale":"2024-05-27T12:35:00.945936+0000","last_undegraded":"2024-05-27T12:35:00.945936+0000","last_fullsized":"2024-05-27T12:35:00.945936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:44.874723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984200+0000","last_change":"2024-05-27T12:34:59.946900+0000","last_active":"2024-05-27T12:35:00.984200+0000","last_peered":"2024-05-27T12:35:00.984200+0000","last_clean":"2024-05-27T12:35:00.984200+0000","last_became_active":"2024-05-27T12:34:59.945045+0000","last_became_peered":"2024-05-27T12:34:59.945045+0000","last_unstale":"2024-05-27T12:35:00.984200+0000","last_undegraded":"2024-05-27T12:35:00.984200+0000","last_fullsized":"2024-05-27T12:35:00.984200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:07:24.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981186+0000","last_change":"2024-05-27T12:34:58.920871+0000","last_active":"2024-05-27T12:35:00.981186+0000","last_peered":"2024-05-27T12:35:00.981186+0000","last_clean":"2024-05-27T12:35:00.981186+0000","last_became_active":"2024-05-27T12:34:58.917367+0000","last_became_peered":"2024-05-27T12:34:58.917367+0000","last_unstale":"2024-05-27T12:35:00.981186+0000","last_undegraded":"2024-05-27T12:35:00.981186+0000","last_fullsized":"2024-05-27T12:35:00.981186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:42:23.759197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.946167+0000","last_change":"2024-05-27T12:34:59.939621+0000","last_active":"2024-05-27T12:35:00.946167+0000","last_peered":"2024-05-27T12:35:00.946167+0000","last_clean":"2024-05-27T12:35:00.946167+0000","last_became_active":"2024-05-27T12:34:59.939269+0000","last_became_peered":"2024-05-27T12:34:59.939269+0000","last_unstale":"2024-05-27T12:35:00.946167+0000","last_undegraded":"2024-05-27T12:35:00.946167+0000","last_fullsized":"2024-05-27T12:35:00.946167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:43:45.879555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.944012+0000","last_change":"2024-05-27T12:34:58.922915+0000","last_active":"2024-05-27T12:35:00.944012+0000","last_peered":"2024-05-27T12:35:00.944012+0000","last_clean":"2024-05-27T12:35:00.944012+0000","last_became_active":"2024-05-27T12:34:58.922438+0000","last_became_peered":"2024-05-27T12:34:58.922438+0000","last_unstale":"2024-05-27T12:35:00.944012+0000","last_undegraded":"2024-05-27T12:35:00.944012+0000","last_fullsized":"2024-05-27T12:35:00.944012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:19:51.784551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980353+0000","last_change":"2024-05-27T12:34:59.947355+0000","last_active":"2024-05-27T12:35:00.980353+0000","last_peered":"2024-05-27T12:35:00.980353+0000","last_clean":"2024-05-27T12:35:00.980353+0000","last_became_active":"2024-05-27T12:34:59.947069+0000","last_became_peered":"2024-05-27T12:34:59.947069+0000","last_unstale":"2024-05-27T12:35:00.980353+0000","last_undegraded":"2024-05-27T12:35:00.980353+0000","last_fullsized":"2024-05-27T12:35:00.980353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:34:04.283425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.943707+0000","last_change":"2024-05-27T12:34:58.911817+0000","last_active":"2024-05-27T12:35:00.943707+0000","last_peered":"2024-05-27T12:35:00.943707+0000","last_clean":"2024-05-27T12:35:00.943707+0000","last_became_active":"2024-05-27T12:34:58.911528+0000","last_became_peered":"2024-05-27T12:34:58.911528+0000","last_unstale":"2024-05-27T12:35:00.943707+0000","last_undegraded":"2024-05-27T12:35:00.943707+0000","last_fullsized":"2024-05-27T12:35:00.943707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:49:26.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981479+0000","last_change":"2024-05-27T12:34:59.944469+0000","last_active":"2024-05-27T12:35:00.981479+0000","last_peered":"2024-05-27T12:35:00.981479+0000","last_clean":"2024-05-27T12:35:00.981479+0000","last_became_active":"2024-05-27T12:34:59.943952+0000","last_became_peered":"2024-05-27T12:34:59.943952+0000","last_unstale":"2024-05-27T12:35:00.981479+0000","last_undegraded":"2024-05-27T12:35:00.981479+0000","last_fullsized":"2024-05-27T12:35:00.981479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:34.962131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982842+0000","last_change":"2024-05-27T12:34:58.928167+0000","last_active":"2024-05-27T12:35:00.982842+0000","last_peered":"2024-05-27T12:35:00.982842+0000","last_clean":"2024-05-27T12:35:00.982842+0000","last_became_active":"2024-05-27T12:34:58.927922+0000","last_became_peered":"2024-05-27T12:34:58.927922+0000","last_unstale":"2024-05-27T12:35:00.982842+0000","last_undegraded":"2024-05-27T12:35:00.982842+0000","last_fullsized":"2024-05-27T12:35:00.982842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:03.179260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.945600+0000","last_change":"2024-05-27T12:34:59.951341+0000","last_active":"2024-05-27T12:35:00.945600+0000","last_peered":"2024-05-27T12:35:00.945600+0000","last_clean":"2024-05-27T12:35:00.945600+0000","last_became_active":"2024-05-27T12:34:59.949217+0000","last_became_peered":"2024-05-27T12:34:59.949217+0000","last_unstale":"2024-05-27T12:35:00.945600+0000","last_undegraded":"2024-05-27T12:35:00.945600+0000","last_fullsized":"2024-05-27T12:35:00.945600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:09.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980544+0000","last_change":"2024-05-27T12:34:59.954080+0000","last_active":"2024-05-27T12:35:00.980544+0000","last_peered":"2024-05-27T12:35:00.980544+0000","last_clean":"2024-05-27T12:35:00.980544+0000","last_became_active":"2024-05-27T12:34:59.946377+0000","last_became_peered":"2024-05-27T12:34:59.946377+0000","last_unstale":"2024-05-27T12:35:00.980544+0000","last_undegraded":"2024-05-27T12:35:00.980544+0000","last_fullsized":"2024-05-27T12:35:00.980544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:53:15.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982176+0000","last_change":"2024-05-27T12:34:58.926815+0000","last_active":"2024-05-27T12:35:00.982176+0000","last_peered":"2024-05-27T12:35:00.982176+0000","last_clean":"2024-05-27T12:35:00.982176+0000","last_became_active":"2024-05-27T12:34:58.926419+0000","last_became_peered":"2024-05-27T12:34:58.926419+0000","last_unstale":"2024-05-27T12:35:00.982176+0000","last_undegraded":"2024-05-27T12:35:00.982176+0000","last_fullsized":"2024-05-27T12:35:00.982176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:05.278200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.980205+0000","last_change":"2024-05-27T12:34:59.942376+0000","last_active":"2024-05-27T12:35:00.980205+0000","last_peered":"2024-05-27T12:35:00.980205+0000","last_clean":"2024-05-27T12:35:00.980205+0000","last_became_active":"2024-05-27T12:34:59.941083+0000","last_became_peered":"2024-05-27T12:34:59.941083+0000","last_unstale":"2024-05-27T12:35:00.980205+0000","last_undegraded":"2024-05-27T12:35:00.980205+0000","last_fullsized":"2024-05-27T12:35:00.980205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:52:14.190264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.982827+0000","last_change":"2024-05-27T12:34:58.927394+0000","last_active":"2024-05-27T12:35:00.982827+0000","last_peered":"2024-05-27T12:35:00.982827+0000","last_clean":"2024-05-27T12:35:00.982827+0000","last_became_active":"2024-05-27T12:34:58.927124+0000","last_became_peered":"2024-05-27T12:34:58.927124+0000","last_unstale":"2024-05-27T12:35:00.982827+0000","last_undegraded":"2024-05-27T12:35:00.982827+0000","last_fullsized":"2024-05-27T12:35:00.982827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:53.128806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984386+0000","last_change":"2024-05-27T12:34:59.946923+0000","last_active":"2024-05-27T12:35:00.984386+0000","last_peered":"2024-05-27T12:35:00.984386+0000","last_clean":"2024-05-27T12:35:00.984386+0000","last_became_active":"2024-05-27T12:34:59.946554+0000","last_became_peered":"2024-05-27T12:34:59.946554+0000","last_unstale":"2024-05-27T12:35:00.984386+0000","last_undegraded":"2024-05-27T12:35:00.984386+0000","last_fullsized":"2024-05-27T12:35:00.984386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:58.222872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981709+0000","last_change":"2024-05-27T12:34:58.912889+0000","last_active":"2024-05-27T12:35:00.981709+0000","last_peered":"2024-05-27T12:35:00.981709+0000","last_clean":"2024-05-27T12:35:00.981709+0000","last_became_active":"2024-05-27T12:34:58.912617+0000","last_became_peered":"2024-05-27T12:34:58.912617+0000","last_unstale":"2024-05-27T12:35:00.981709+0000","last_undegraded":"2024-05-27T12:35:00.981709+0000","last_fullsized":"2024-05-27T12:35:00.981709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:32:16.022007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.981440+0000","last_change":"2024-05-27T12:34:59.941269+0000","last_active":"2024-05-27T12:35:00.981440+0000","last_peered":"2024-05-27T12:35:00.981440+0000","last_clean":"2024-05-27T12:35:00.981440+0000","last_became_active":"2024-05-27T12:34:59.941004+0000","last_became_peered":"2024-05-27T12:34:59.941004+0000","last_unstale":"2024-05-27T12:35:00.981440+0000","last_undegraded":"2024-05-27T12:35:00.981440+0000","last_fullsized":"2024-05-27T12:35:00.981440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:58.874373+0000","last_clean_scrub_stamp":"2024-05-27T12:34:58.874373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:46:29.906845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-27T12:35:00.984278+0000","last_change":"2024-05-27T12:34:58.918917+0000","last_active":"2024-05-27T12:35:00.984278+0000","last_peered":"2024-05-27T12:35:00.984278+0000","last_clean":"2024-05-27T12:35:00.984278+0000","last_became_active":"2024-05-27T12:34:58.918499+0000","last_became_peered":"2024-05-27T12:34:58.918499+0000","last_unstale":"2024-05-27T12:35:00.984278+0000","last_undegraded":"2024-05-27T12:35:00.984278+0000","last_fullsized":"2024-05-27T12:35:00.984278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:57.869542+0000","last_clean_scrub_stamp":"2024-05-27T12:34:57.869542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:13.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:35:08.439 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-27T12:35:08.439 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T12:35:08.439 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-27T12:35:08.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-27T12:35:08.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.670+0000 7ff31a200640 1 Processor -- start 2024-05-27T12:35:08.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.670+0000 7ff31a200640 1 -- start start 2024-05-27T12:35:08.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff31a200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 0x7ff314071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff31a200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff31406eed0 0x7ff31406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff31a200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31413e9e0 con 0x7ff3140cd480 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff31a200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff31415ba50 con 0x7ff31406eed0 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff31a200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff31415bd10 con 0x7ff31413e600 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff319800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff3140cd480 0x7ff3140cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:48140/0 (socket says 172.21.15.94:48140) 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff313e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff31406eed0 0x7ff31406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.672+0000 7ff319800640 1 -- 172.21.15.94:0/2099135853 learned_addr learned my addr 172.21.15.94:0/2099135853 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff318e00640 1 --2- 172.21.15.94:0/2099135853 >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 0x7ff314071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff313e00640 1 -- 172.21.15.94:0/2099135853 >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 msgr2=0x7ff314071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff313e00640 1 --2- 172.21.15.94:0/2099135853 >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 0x7ff314071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff313e00640 1 -- 172.21.15.94:0/2099135853 >> v1:172.21.15.94:6789/0 conn(0x7ff3140cd480 legacy=0x7ff3140cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff313e00640 1 -- 172.21.15.94:0/2099135853 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31415bff0 con 0x7ff31406eed0 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.673+0000 7ff318e00640 1 --2- 172.21.15.94:0/2099135853 >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 0x7ff314071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:08.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff313e00640 1 --2- 172.21.15.94:0/2099135853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31406f380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff31406d660 tx=0x7ff30002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a0a422226d7c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff313400640 1 -- 172.21.15.94:0/2099135853 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300037070 con 0x7ff31406eed0 2024-05-27T12:35:08.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff313400640 1 -- 172.21.15.94:0/2099135853 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff300032440 con 0x7ff31406eed0 2024-05-27T12:35:08.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff313400640 1 -- 172.21.15.94:0/2099135853 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300032810 con 0x7ff31406eed0 2024-05-27T12:35:08.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff31a200640 1 -- 172.21.15.94:0/2099135853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 msgr2=0x7ff31406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.674+0000 7ff31a200640 1 --2- 172.21.15.94:0/2099135853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31406f380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff31406d660 tx=0x7ff30002a1a0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 -- 172.21.15.94:0/2099135853 shutdown_connections 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 --2- 172.21.15.94:0/2099135853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31406f380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 --2- 172.21.15.94:0/2099135853 >> v2:172.21.15.97:3300/0 conn(0x7ff31413e600 0x7ff314071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 -- 172.21.15.94:0/2099135853 >> 172.21.15.94:0/2099135853 conn(0x7ff3140b2160 msgr2=0x7ff3140b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 -- 172.21.15.94:0/2099135853 shutdown_connections 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.675+0000 7ff31a200640 1 -- 172.21.15.94:0/2099135853 wait complete. 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.676+0000 7ff31a200640 1 Processor -- start 2024-05-27T12:35:08.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.676+0000 7ff31a200640 1 -- start start 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3140cd480 0x7ff314159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 0x7ff314157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff31415c920 con 0x7ff31406eed0 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff31415c5b0 con 0x7ff31413e600 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff31a200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff314141030 con 0x7ff3140cd480 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff318e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff318e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56736/0 (socket says 172.21.15.94:56736) 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.677+0000 7ff318e00640 1 -- 172.21.15.94:0/2058870745 learned_addr learned my addr 172.21.15.94:0/2058870745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 -- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3140cd480 msgr2=0x7ff314159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3140cd480 0x7ff314159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 -- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 msgr2=0x7ff314157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff319800640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 0x7ff314157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 0x7ff314157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 -- 172.21.15.94:0/2058870745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff314157000 con 0x7ff31406eed0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff319800640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 0x7ff314157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff318e00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c820 tx=0x7ff30800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308015070 con 0x7ff31406eed0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff308007a30 con 0x7ff31406eed0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff314151d00 con 0x7ff31406eed0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.678+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308007de0 con 0x7ff31406eed0 2024-05-27T12:35:08.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.679+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3141522c0 con 0x7ff31406eed0 2024-05-27T12:35:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.680+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff314075460 con 0x7ff31406eed0 2024-05-27T12:35:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.680+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff308018070 con 0x7ff31406eed0 2024-05-27T12:35:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.681+0000 7ff310c00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 0x7ff2f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:08.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.681+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff30808ba50 con 0x7ff31406eed0 2024-05-27T12:35:08.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.682+0000 7ff313e00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 0x7ff2f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:08.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.683+0000 7ff313e00640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 0x7ff2f406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff300034000 tx=0x7ff30002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:08.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.685+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff308059e70 con 0x7ff31406eed0 2024-05-27T12:35:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.908+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff3140cfbf0 con 0x7ff31406eed0 2024-05-27T12:35:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.908+0000 7ff310c00640 1 -- 172.21.15.94:0/2058870745 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff308059770 con 0x7ff31406eed0 2024-05-27T12:35:08.908 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:08.908 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T12:35:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 msgr2=0x7ff2f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 0x7ff2f406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff300034000 tx=0x7ff30002c000 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 msgr2=0x7ff31413d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c820 tx=0x7ff30800cce0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 shutdown_connections 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff2f406c8f0 0x7ff2f406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31413e600 0x7ff314157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3140cd480 0x7ff314159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 --2- 172.21.15.94:0/2058870745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff31406eed0 0x7ff31413d1e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 >> 172.21.15.94:0/2058870745 conn(0x7ff3140b2160 msgr2=0x7ff31415cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 shutdown_connections 2024-05-27T12:35:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:08.911+0000 7ff31a200640 1 -- 172.21.15.94:0/2058870745 wait complete. 2024-05-27T12:35:08.928 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-27T12:35:08.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.127+0000 7f1a91e00640 1 Processor -- start 2024-05-27T12:35:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.128+0000 7f1a91e00640 1 -- start start 2024-05-27T12:35:09.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.129+0000 7f1a91e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.129+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 0x7f1a8c1684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.129+0000 7f1a91e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c15fdd0 con 0x7f1a8c15eb00 2024-05-27T12:35:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.129+0000 7f1a91e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c0d08b0 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.129+0000 7f1a91e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c0d0b50 con 0x7f1a8c09b140 2024-05-27T12:35:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.130+0000 7f1a82000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.130+0000 7f1a8b400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 0x7f1a8c1684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.130+0000 7f1a82000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59384/0 (socket says 172.21.15.94:59384) 2024-05-27T12:35:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.130+0000 7f1a82000640 1 -- 172.21.15.94:0/333830978 learned_addr learned my addr 172.21.15.94:0/333830978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:09.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.131+0000 7f1a82000640 1 -- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 msgr2=0x7f1a8c1684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.131+0000 7f1a82000640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 0x7f1a8c1684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.131+0000 7f1a82000640 1 -- 172.21.15.94:0/333830978 >> v1:172.21.15.97:6789/0 conn(0x7f1a8c15eb00 legacy=0x7f1a8c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.131+0000 7f1a82000640 1 -- 172.21.15.94:0/333830978 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0d0e30 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.131+0000 7f1a8b400640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 0x7f1a8c1684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:35:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.132+0000 7f1a82000640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009870 tx=0x7f1a7402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc79cbc2ceb35eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a8a000640 1 -- 172.21.15.94:0/333830978 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74037070 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a8a000640 1 -- 172.21.15.94:0/333830978 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7402ae50 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a8a000640 1 -- 172.21.15.94:0/333830978 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74032860 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 -- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 msgr2=0x7f1a8c0d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009870 tx=0x7f1a7402a300 comp rx=0 tx=0).stop 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 -- 172.21.15.94:0/333830978 shutdown_connections 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c09b140 0x7f1a8c1684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 --2- 172.21.15.94:0/333830978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c0d1630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.133+0000 7f1a91e00640 1 -- 172.21.15.94:0/333830978 >> 172.21.15.94:0/333830978 conn(0x7f1a8c0b4100 msgr2=0x7f1a8c0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.134+0000 7f1a91e00640 1 -- 172.21.15.94:0/333830978 shutdown_connections 2024-05-27T12:35:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.134+0000 7f1a91e00640 1 -- 172.21.15.94:0/333830978 wait complete. 2024-05-27T12:35:09.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.136+0000 7f1a91e00640 1 Processor -- start 2024-05-27T12:35:09.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.136+0000 7f1a91e00640 1 -- start start 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c15eb00 0x7f1a8c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c1f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c09b5d0 con 0x7f1a8c09b140 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c160040 con 0x7f1a8c15f9b0 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.137+0000 7f1a91e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c0b6110 con 0x7f1a8c15eb00 2024-05-27T12:35:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56752/0 (socket says 172.21.15.94:56752) 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 -- 172.21.15.94:0/2912617518 learned_addr learned my addr 172.21.15.94:0/2912617518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a82000640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c15eb00 0x7f1a8c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8b400640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c1f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 -- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c15eb00 msgr2=0x7f1a8c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c15eb00 0x7f1a8c0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 -- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 msgr2=0x7f1a8c1f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c1f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 -- 172.21.15.94:0/2912617518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0b5500 con 0x7f1a8c09b140 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8b400640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c1f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.138+0000 7f1a8aa00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c430 tx=0x7f1a7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.139+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c017070 con 0x7f1a8c09b140 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.139+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7c0052d0 con 0x7f1a8c09b140 2024-05-27T12:35:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.139+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c003830 con 0x7f1a8c09b140 2024-05-27T12:35:09.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.139+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c0b57e0 con 0x7f1a8c09b140 2024-05-27T12:35:09.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.139+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c0c8eb0 con 0x7f1a8c09b140 2024-05-27T12:35:09.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.140+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c075460 con 0x7f1a8c09b140 2024-05-27T12:35:09.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.141+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a7c01a070 con 0x7f1a8c09b140 2024-05-27T12:35:09.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.142+0000 7f1a83e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 0x7f1a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.142+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7c08af30 con 0x7f1a8c09b140 2024-05-27T12:35:09.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.143+0000 7f1a82000640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 0x7f1a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.144+0000 7f1a82000640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 0x7f1a6806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a74034000 tx=0x7f1a7402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.148+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a7c0592c0 con 0x7f1a8c09b140 2024-05-27T12:35:09.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.351+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a8c0b59b0 con 0x7f1a8c09b140 2024-05-27T12:35:09.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.354+0000 7f1a83e00640 1 -- 172.21.15.94:0/2912617518 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f1a7c058bc0 con 0x7f1a8c09b140 2024-05-27T12:35:09.354 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:09.354 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.356+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 msgr2=0x7f1a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.356+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 0x7f1a6806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a74034000 tx=0x7f1a7402e000 comp rx=0 tx=0).stop 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.356+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 msgr2=0x7f1a8c0b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.356+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c430 tx=0x7f1a7c00c8f0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 shutdown_connections 2024-05-27T12:35:09.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f1a6806c8f0 0x7f1a6806edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a8c15f9b0 0x7f1a8c1f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a8c15eb00 0x7f1a8c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 --2- 172.21.15.94:0/2912617518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a8c09b140 0x7f1a8c0b65a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 >> 172.21.15.94:0/2912617518 conn(0x7f1a8c0b4100 msgr2=0x7f1a8c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 shutdown_connections 2024-05-27T12:35:09.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.357+0000 7f1a91e00640 1 -- 172.21.15.94:0/2912617518 wait complete. 2024-05-27T12:35:09.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:09.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:09.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.575+0000 7f4d91200640 1 Processor -- start 2024-05-27T12:35:09.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.575+0000 7f4d91200640 1 -- start start 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d91200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d91200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d91200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c15eee0 con 0x7f4d8c0d1d70 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d91200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8c0d5700 con 0x7f4d8c15eb00 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d91200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c0d21a0 con 0x7f4d8c0d3e50 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d89e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d89e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56766/0 (socket says 172.21.15.94:56766) 2024-05-27T12:35:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.577+0000 7f4d89e00640 1 -- 172.21.15.94:0/612790341 learned_addr learned my addr 172.21.15.94:0/612790341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.578+0000 7f4d89400640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.578+0000 7f4d89e00640 1 -- 172.21.15.94:0/612790341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 msgr2=0x7f4d8c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.578+0000 7f4d89e00640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.578+0000 7f4d89e00640 1 -- 172.21.15.94:0/612790341 >> v1:172.21.15.97:6789/0 conn(0x7f4d8c0d1d70 legacy=0x7f4d8c0d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.578+0000 7f4d89e00640 1 -- 172.21.15.94:0/612790341 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c0d4660 con 0x7f4d8c15eb00 2024-05-27T12:35:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.579+0000 7f4d89e00640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009e80 tx=0x7f4d7c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2968699e663b5117 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.579+0000 7f4d88a00640 1 -- 172.21.15.94:0/612790341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c036070 con 0x7f4d8c15eb00 2024-05-27T12:35:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.579+0000 7f4d88a00640 1 -- 172.21.15.94:0/612790341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c031600 con 0x7f4d8c15eb00 2024-05-27T12:35:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.579+0000 7f4d88a00640 1 -- 172.21.15.94:0/612790341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0319d0 con 0x7f4d8c15eb00 2024-05-27T12:35:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 -- 172.21.15.94:0/612790341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 msgr2=0x7f4d8c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009e80 tx=0x7f4d7c02a780 comp rx=0 tx=0).stop 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 -- 172.21.15.94:0/612790341 shutdown_connections 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 --2- 172.21.15.94:0/612790341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c0cfb40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 -- 172.21.15.94:0/612790341 >> 172.21.15.94:0/612790341 conn(0x7f4d8c067a50 msgr2=0x7f4d8c067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 -- 172.21.15.94:0/612790341 shutdown_connections 2024-05-27T12:35:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.580+0000 7f4d91200640 1 -- 172.21.15.94:0/612790341 wait complete. 2024-05-27T12:35:09.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.582+0000 7f4d91200640 1 Processor -- start 2024-05-27T12:35:09.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.582+0000 7f4d91200640 1 -- start start 2024-05-27T12:35:09.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c0d1d70 0x7f4d8c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c0d4f00 con 0x7f4d8c15eb00 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c0d4c20 con 0x7f4d8c0d1d70 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d91200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c0d51e0 con 0x7f4d8c0d3e50 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d8a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d8a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56782/0 (socket says 172.21.15.94:56782) 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.583+0000 7f4d8a800640 1 -- 172.21.15.94:0/2730058758 learned_addr learned my addr 172.21.15.94:0/2730058758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d89400640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d89e00640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c0d1d70 0x7f4d8c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 -- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 msgr2=0x7f4d8c1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c1f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 -- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c0d1d70 msgr2=0x7f4d8c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c0d1d70 0x7f4d8c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 -- 172.21.15.94:0/2730058758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c0cd760 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d8a800640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c410 tx=0x7f4d8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84011790 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84017070 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c0cda40 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84011bb0 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.584+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c0ce000 con 0x7f4d8c15eb00 2024-05-27T12:35:09.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.585+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c068a70 con 0x7f4d8c15eb00 2024-05-27T12:35:09.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.587+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d8401c460 con 0x7f4d8c15eb00 2024-05-27T12:35:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.588+0000 7f4d82000640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 0x7f4d6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.588+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d8408b810 con 0x7f4d8c15eb00 2024-05-27T12:35:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.588+0000 7f4d89e00640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 0x7f4d6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.590+0000 7f4d89e00640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 0x7f4d6806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c033000 tx=0x7f4d7c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.592+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d84021020 con 0x7f4d8c15eb00 2024-05-27T12:35:09.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.798+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d8c0d33c0 con 0x7f4d8c15eb00 2024-05-27T12:35:09.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.799+0000 7f4d82000640 1 -- 172.21.15.94:0/2730058758 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4d84059550 con 0x7f4d8c15eb00 2024-05-27T12:35:09.798 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 msgr2=0x7f4d6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 0x7f4d6806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c033000 tx=0x7f4d7c009000 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 msgr2=0x7f4d8c1f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c410 tx=0x7f4d8400c8d0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 shutdown_connections 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4d6806c920 0x7f4d6806ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d8c15eb00 0x7f4d8c1f4180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d8c0d3e50 0x7f4d8c1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 --2- 172.21.15.94:0/2730058758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d8c0d1d70 0x7f4d8c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 >> 172.21.15.94:0/2730058758 conn(0x7f4d8c067a50 msgr2=0x7f4d8c0b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 shutdown_connections 2024-05-27T12:35:09.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:09.801+0000 7f4d91200640 1 -- 172.21.15.94:0/2730058758 wait complete. 2024-05-27T12:35:09.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.026+0000 7fa513200640 1 Processor -- start 2024-05-27T12:35:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.027+0000 7fa513200640 1 -- start start 2024-05-27T12:35:10.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa513200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa51415c790 0x7fa514160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa513200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa513200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51415cb70 con 0x7fa5140b3a00 2024-05-27T12:35:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa513200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa5140d0c20 con 0x7fa51415c790 2024-05-27T12:35:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa513200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa5141611d0 con 0x7fa5140cf5c0 2024-05-27T12:35:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa511400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa5140b3a00 0x7fa5140ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50310/0 (socket says 172.21.15.94:50310) 2024-05-27T12:35:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa511400640 1 -- 172.21.15.94:0/516256114 learned_addr learned my addr 172.21.15.94:0/516256114 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.028+0000 7fa511e00640 1 --2- 172.21.15.94:0/516256114 >> v2:172.21.15.94:3300/0 conn(0x7fa51415c790 0x7fa514160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa512800640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa511e00640 1 -- 172.21.15.94:0/516256114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 msgr2=0x7fa5141665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa511e00640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa511e00640 1 -- 172.21.15.94:0/516256114 >> v1:172.21.15.94:6789/0 conn(0x7fa5140b3a00 legacy=0x7fa5140ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa510a00640 1 -- 172.21.15.94:0/516256114 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952578728 0 0) 0x7fa51415cb70 con 0x7fa5140b3a00 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa511e00640 1 -- 172.21.15.94:0/516256114 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140b2960 con 0x7fa51415c790 2024-05-27T12:35:10.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.029+0000 7fa512800640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.030+0000 7fa511e00640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa51415c790 0x7fa514160c90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097d0 tx=0x7fa50402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a3e98c8be7203c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.030+0000 7fa510a00640 1 -- 172.21.15.94:0/516256114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504037070 con 0x7fa51415c790 2024-05-27T12:35:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.030+0000 7fa510a00640 1 -- 172.21.15.94:0/516256114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504032440 con 0x7fa51415c790 2024-05-27T12:35:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.030+0000 7fa510a00640 1 -- 172.21.15.94:0/516256114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504032810 con 0x7fa51415c790 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 -- 172.21.15.94:0/516256114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa51415c790 msgr2=0x7fa514160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa51415c790 0x7fa514160c90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097d0 tx=0x7fa50402a1c0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 -- 172.21.15.94:0/516256114 shutdown_connections 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 --2- 172.21.15.94:0/516256114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa51415c790 0x7fa514160c90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 -- 172.21.15.94:0/516256114 >> 172.21.15.94:0/516256114 conn(0x7fa5140b3fb0 msgr2=0x7fa5140b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 -- 172.21.15.94:0/516256114 shutdown_connections 2024-05-27T12:35:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.031+0000 7fa513200640 1 -- 172.21.15.94:0/516256114 wait complete. 2024-05-27T12:35:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.033+0000 7fa513200640 1 Processor -- start 2024-05-27T12:35:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.033+0000 7fa513200640 1 -- start start 2024-05-27T12:35:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa51415c790 0x7fa5140c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b3200 con 0x7fa5140b3a00 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b2f20 con 0x7fa51415c790 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa513200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa5140b34e0 con 0x7fa5140cf5c0 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37092/0 (socket says 172.21.15.94:37092) 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 -- 172.21.15.94:0/3336497895 learned_addr learned my addr 172.21.15.94:0/3336497895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 -- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 msgr2=0x7fa5141508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 -- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa51415c790 msgr2=0x7fa5140c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa51415c790 0x7fa5140c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.034+0000 7fa511e00640 1 -- 172.21.15.94:0/3336497895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140b02e0 con 0x7fa5140b3a00 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa511e00640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097d0 tx=0x7fa5040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504037050 con 0x7fa5140b3a00 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504038040 con 0x7fa5140b3a00 2024-05-27T12:35:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5140b05c0 con 0x7fa5140b3a00 2024-05-27T12:35:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504004470 con 0x7fa5140b3a00 2024-05-27T12:35:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.035+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5140b0b80 con 0x7fa5140b3a00 2024-05-27T12:35:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.036+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa514075460 con 0x7fa5140b3a00 2024-05-27T12:35:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.037+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa504030980 con 0x7fa5140b3a00 2024-05-27T12:35:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.038+0000 7fa50a000640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.038+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5040ac7a0 con 0x7fa5140b3a00 2024-05-27T12:35:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.039+0000 7fa511400640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.040+0000 7fa511400640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc001560 tx=0x7fa4fc007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.043+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa50407abd0 con 0x7fa5140b3a00 2024-05-27T12:35:10.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.245+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa51409bed0 con 0x7fa5140b3a00 2024-05-27T12:35:10.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.246+0000 7fa50a000640 1 -- 172.21.15.94:0/3336497895 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa50407a4d0 con 0x7fa5140b3a00 2024-05-27T12:35:10.246 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:10.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 msgr2=0x7fa4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc001560 tx=0x7fa4fc007ca0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 msgr2=0x7fa5140cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5040097d0 tx=0x7fa5040046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 shutdown_connections 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa51415c790 0x7fa5140c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5140cf5c0 0x7fa5141508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 --2- 172.21.15.94:0/3336497895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5140b3a00 0x7fa5140cbf60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 >> 172.21.15.94:0/3336497895 conn(0x7fa5140b3fb0 msgr2=0x7fa514160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 shutdown_connections 2024-05-27T12:35:10.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.249+0000 7fa513200640 1 -- 172.21.15.94:0/3336497895 wait complete. 2024-05-27T12:35:10.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:10.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.470+0000 7f46fb200640 1 Processor -- start 2024-05-27T12:35:10.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.470+0000 7f46fb200640 1 -- start start 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.471+0000 7f46fb200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46fc0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.471+0000 7f46fb200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc0cf2f0 con 0x7f46fc145020 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.471+0000 7f46fb200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc0cf590 con 0x7f46fc06ed60 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.471+0000 7f46fb200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc0cf870 con 0x7f46fc0afba0 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f9400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f46fc145020 0x7f46fc0b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50324/0 (socket says 172.21.15.94:50324) 2024-05-27T12:35:10.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f9400640 1 -- 172.21.15.94:0/2815618964 learned_addr learned my addr 172.21.15.94:0/2815618964 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46fa800640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46fc0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644798559 0 0) 0x7f46fc0cf2f0 con 0x7f46fc145020 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e0005730 con 0x7f46fc145020 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4121734201 0 0) 0x7f46e0005730 con 0x7f46fc145020 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 msgr2=0x7f46fc0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46fc0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 >> v1:172.21.15.97:6789/0 conn(0x7f46fc06ed60 legacy=0x7f46fc06f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.472+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc0cfb50 con 0x7f46fc145020 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f46e8001670 con 0x7f46fc145020 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 >> v1:172.21.15.94:6789/0 conn(0x7f46fc145020 legacy=0x7f46fc0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46e0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46e0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e0006f50 0x7f46e00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc0cf870 con 0x7f46e0006220 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46e000de10 con 0x7f46e0006f50 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f46e000e010 con 0x7f46fc0afba0 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46e0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9e00640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46e0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46fa800640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e0006f50 0x7f46e00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 -- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 msgr2=0x7f46e0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc0afba0 0x7f46e0005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 -- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e0006f50 msgr2=0x7f46e00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e0006f50 0x7f46e00073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46f9400640 1 -- 172.21.15.94:0/2815618964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0005730 con 0x7f46e0006220 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.473+0000 7f46fa800640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e0006f50 0x7f46e00073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.474+0000 7f46f9400640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46e0006a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c240 tx=0x7f46e0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941179a9be518c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007d10 con 0x7f46e0006220 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8005ad0 con 0x7f46e0006220 2024-05-27T12:35:10.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46f8a00640 1 -- 172.21.15.94:0/2815618964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007070 con 0x7f46e0006220 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46fb200640 1 -- 172.21.15.94:0/2815618964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 msgr2=0x7f46e0006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46fb200640 1 --2- 172.21.15.94:0/2815618964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46e0006a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c240 tx=0x7f46e0007a40 comp rx=0 tx=0).stop 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46fa800640 1 -- 172.21.15.94:0/2815618964 reap_dead start 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46fb200640 1 -- 172.21.15.94:0/2815618964 shutdown_connections 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.475+0000 7f46fb200640 1 -- 172.21.15.94:0/2815618964 >> 172.21.15.94:0/2815618964 conn(0x7f46fc0b6230 msgr2=0x7f46fc0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.476+0000 7f46fb200640 1 -- 172.21.15.94:0/2815618964 shutdown_connections 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.476+0000 7f46fb200640 1 -- 172.21.15.94:0/2815618964 wait complete. 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.477+0000 7f46fb200640 1 Processor -- start 2024-05-27T12:35:10.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.477+0000 7f46fb200640 1 -- start start 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.478+0000 7f46fb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.478+0000 7f46fb200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc155260 0x7f46fc13a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.478+0000 7f46fb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46fc06dc30 0x7f46fc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.478+0000 7f46fb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc144710 con 0x7f46e0006220 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.478+0000 7f46fb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc144590 con 0x7f46fc06dc30 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46fb200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc144980 con 0x7f46fc155260 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37104/0 (socket says 172.21.15.94:37104) 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 -- 172.21.15.94:0/964744952 learned_addr learned my addr 172.21.15.94:0/964744952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9400640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc155260 0x7f46fc13a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 -- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc155260 msgr2=0x7f46fc13a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc155260 0x7f46fc13a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46fa800640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46fc06dc30 0x7f46fc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 -- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46fc06dc30 msgr2=0x7f46fc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46fc06dc30 0x7f46fc06e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 -- 172.21.15.94:0/964744952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc0ae800 con 0x7f46e0006220 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.479+0000 7f46f9e00640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e5b0 tx=0x7f46ec00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.480+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0052e0 con 0x7f46e0006220 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.480+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec018070 con 0x7f46e0006220 2024-05-27T12:35:10.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.480+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003810 con 0x7f46e0006220 2024-05-27T12:35:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.480+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc0ada40 con 0x7f46e0006220 2024-05-27T12:35:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.480+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc0ae000 con 0x7f46e0006220 2024-05-27T12:35:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.481+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc075460 con 0x7f46e0006220 2024-05-27T12:35:10.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.482+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46ec0039b0 con 0x7f46e0006220 2024-05-27T12:35:10.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.483+0000 7f46f2000640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.483+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46ec08b7f0 con 0x7f46e0006220 2024-05-27T12:35:10.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.483+0000 7f46f9400640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.485+0000 7f46f9400640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 0x7f46dc06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46e8033000 tx=0x7f46e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.487+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46ec059b80 con 0x7f46e0006220 2024-05-27T12:35:10.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.688+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46fc15e1e0 con 0x7f46e0006220 2024-05-27T12:35:10.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.689+0000 7f46f2000640 1 -- 172.21.15.94:0/964744952 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f46ec059480 con 0x7f46e0006220 2024-05-27T12:35:10.689 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 msgr2=0x7f46dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 0x7f46dc06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46e8033000 tx=0x7f46e802e000 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 msgr2=0x7f46fc0af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e5b0 tx=0x7f46ec00ea70 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 shutdown_connections 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f46dc06c8f0 0x7f46dc06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46fc06dc30 0x7f46fc06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46fc155260 0x7f46fc13a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 --2- 172.21.15.94:0/964744952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46e0006220 0x7f46fc0af210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 >> 172.21.15.94:0/964744952 conn(0x7f46fc0b6230 msgr2=0x7f46fc0cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 shutdown_connections 2024-05-27T12:35:10.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.692+0000 7f46fb200640 1 -- 172.21.15.94:0/964744952 wait complete. 2024-05-27T12:35:10.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:10.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.910+0000 7f88f2400640 1 Processor -- start 2024-05-27T12:35:10.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.910+0000 7f88f2400640 1 -- start start 2024-05-27T12:35:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88f2400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88ec071f20 0x7f88ec072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88f2400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88ec0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88f2400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ec0ce200 con 0x7f88ec144bb0 2024-05-27T12:35:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88f2400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f88ec144f90 con 0x7f88ec071f20 2024-05-27T12:35:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88f2400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec0728c0 con 0x7f88ec0cf040 2024-05-27T12:35:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88eb400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f88ec144bb0 0x7f88ec0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50334/0 (socket says 172.21.15.94:50334) 2024-05-27T12:35:10.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.912+0000 7f88eb400640 1 -- 172.21.15.94:0/3650740668 learned_addr learned my addr 172.21.15.94:0/3650740668 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88e3e00640 1 --2- 172.21.15.94:0/3650740668 >> v2:172.21.15.97:3300/0 conn(0x7f88ec071f20 0x7f88ec072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88ebe00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88ec0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184399936 0 0) 0x7f88ec0ce200 con 0x7f88ec144bb0 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88d0003520 con 0x7f88ec144bb0 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 674839938 0 0) 0x7f88d0003520 con 0x7f88ec144bb0 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 msgr2=0x7f88ec0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88ec0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 >> v2:172.21.15.97:3300/0 conn(0x7f88ec071f20 msgr2=0x7f88ec072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 --2- 172.21.15.94:0/3650740668 >> v2:172.21.15.97:3300/0 conn(0x7f88ec071f20 0x7f88ec072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec0cdc40 con 0x7f88ec144bb0 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f88dc001670 con 0x7f88ec144bb0 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 >> v1:172.21.15.94:6789/0 conn(0x7f88ec144bb0 legacy=0x7f88ec0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88d0003a90 0x7f88d0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec144f90 con 0x7f88d0004d40 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88d000baa0 con 0x7f88d0003a90 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.913+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f88d000bca0 con 0x7f88ec0cf040 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88e3e00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88eb400640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88d0003a90 0x7f88d0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 -- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 msgr2=0x7f88d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec0cf040 0x7f88d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 -- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88d0003a90 msgr2=0x7f88d0003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88d0003a90 0x7f88d0003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88ebe00640 1 -- 172.21.15.94:0/3650740668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0003520 con 0x7f88d0004d40 2024-05-27T12:35:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.914+0000 7f88eb400640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88d0003a90 0x7f88d0003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.915+0000 7f88ebe00640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88d00051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88e4007330 tx=0x7f88d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8bc868fbf762fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.915+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4033070 con 0x7f88d0004d40 2024-05-27T12:35:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.915+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e402f650 con 0x7f88d0004d40 2024-05-27T12:35:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.915+0000 7f88eaa00640 1 -- 172.21.15.94:0/3650740668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4009be0 con 0x7f88d0004d40 2024-05-27T12:35:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 -- 172.21.15.94:0/3650740668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 msgr2=0x7f88d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 --2- 172.21.15.94:0/3650740668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88d00051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88e4007330 tx=0x7f88d0005830 comp rx=0 tx=0).stop 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88ebe00640 1 -- 172.21.15.94:0/3650740668 reap_dead start 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 -- 172.21.15.94:0/3650740668 shutdown_connections 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 -- 172.21.15.94:0/3650740668 >> 172.21.15.94:0/3650740668 conn(0x7f88ec0b3830 msgr2=0x7f88ec0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 -- 172.21.15.94:0/3650740668 shutdown_connections 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.916+0000 7f88f2400640 1 -- 172.21.15.94:0/3650740668 wait complete. 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.918+0000 7f88f2400640 1 Processor -- start 2024-05-27T12:35:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.918+0000 7f88f2400640 1 -- start start 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ec1dbcd0 0x7f88ec1e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec1e09e0 0x7f88ec1319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec0ceb20 con 0x7f88d0004d40 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec0ce840 con 0x7f88ec1dbcd0 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88f2400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec145330 con 0x7f88ec1e09e0 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37122/0 (socket says 172.21.15.94:37122) 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.919+0000 7f88eb400640 1 -- 172.21.15.94:0/2969953333 learned_addr learned my addr 172.21.15.94:0/2969953333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 -- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec1e09e0 msgr2=0x7f88ec1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec1e09e0 0x7f88ec1319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 -- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ec1dbcd0 msgr2=0x7f88ec1e04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ec1dbcd0 0x7f88ec1e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 -- 172.21.15.94:0/2969953333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1ded10 con 0x7f88d0004d40 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88eb400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00d810 tx=0x7f88dc00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc019070 con 0x7f88d0004d40 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc0074e0 con 0x7f88d0004d40 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1deff0 con 0x7f88d0004d40 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0049e0 con 0x7f88d0004d40 2024-05-27T12:35:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.920+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1df5b0 con 0x7f88d0004d40 2024-05-27T12:35:10.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.922+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec075460 con 0x7f88d0004d40 2024-05-27T12:35:10.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.922+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88dc01c070 con 0x7f88d0004d40 2024-05-27T12:35:10.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.923+0000 7f88e3400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 0x7f88c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:10.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.923+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f88dc08b540 con 0x7f88d0004d40 2024-05-27T12:35:10.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.924+0000 7f88e3e00640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 0x7f88c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:10.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.925+0000 7f88e3e00640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 0x7f88c806ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0b5a40 tx=0x7f88d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:10.927+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88dc059a00 con 0x7f88d0004d40 2024-05-27T12:35:11.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.153+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88ec1e58a0 con 0x7f88d0004d40 2024-05-27T12:35:11.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.154+0000 7f88e3400640 1 -- 172.21.15.94:0/2969953333 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f88dc059300 con 0x7f88d0004d40 2024-05-27T12:35:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:11.154 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:11.154 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 msgr2=0x7f88c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 0x7f88c806ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0b5a40 tx=0x7f88d4008040 comp rx=0 tx=0).stop 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 msgr2=0x7f88ec0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00d810 tx=0x7f88dc00dcd0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 shutdown_connections 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f88c806c920 0x7f88c806ede0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ec1dbcd0 0x7f88ec1e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88ec1e09e0 0x7f88ec1319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 --2- 172.21.15.94:0/2969953333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d0004d40 0x7f88ec0b50f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.156+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 >> 172.21.15.94:0/2969953333 conn(0x7f88ec0b3830 msgr2=0x7f88ec1e3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.157+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 shutdown_connections 2024-05-27T12:35:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.157+0000 7f88f2400640 1 -- 172.21.15.94:0/2969953333 wait complete. 2024-05-27T12:35:11.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:11.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.372+0000 7f6b9b200640 1 Processor -- start 2024-05-27T12:35:11.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.372+0000 7f6b9b200640 1 -- start start 2024-05-27T12:35:11.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b9b200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6b9415ec00 0x7f6b940b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b9b200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b941632f0 con 0x7f6b940b3860 2024-05-27T12:35:11.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b9b200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b941635d0 con 0x7f6b940b6370 2024-05-27T12:35:11.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b9b200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6b940cf2e0 con 0x7f6b9415ec00 2024-05-27T12:35:11.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6b9415ec00 0x7f6b940b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6b940b3860 0x7f6b940cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50340/0 (socket says 172.21.15.94:50340) 2024-05-27T12:35:11.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93400640 1 -- 172.21.15.94:0/3104945820 learned_addr learned my addr 172.21.15.94:0/3104945820 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93e00640 1 -- 172.21.15.94:0/3104945820 >> v1:172.21.15.97:6789/0 conn(0x7f6b940b6370 legacy=0x7f6b940d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93e00640 1 -- 172.21.15.94:0/3104945820 >> v1:172.21.15.94:6789/0 conn(0x7f6b940b3860 legacy=0x7f6b940cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b93e00640 1 -- 172.21.15.94:0/3104945820 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b940cf530 con 0x7f6b9415ec00 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.374+0000 7f6b92a00640 1 -- 172.21.15.94:0/3104945820 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839521068 0 0) 0x7f6b941632f0 con 0x7f6b940b3860 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.375+0000 7f6b93e00640 1 --2- 172.21.15.94:0/3104945820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b9415ec00 0x7f6b940b3db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b840097a0 tx=0x7f6b8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4e18ea0355a23d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b92a00640 1 -- 172.21.15.94:0/3104945820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84037070 con 0x7f6b9415ec00 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b92a00640 1 -- 172.21.15.94:0/3104945820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84032440 con 0x7f6b9415ec00 2024-05-27T12:35:11.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b92a00640 1 -- 172.21.15.94:0/3104945820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84032810 con 0x7f6b9415ec00 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 -- 172.21.15.94:0/3104945820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b9415ec00 msgr2=0x7f6b940b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 --2- 172.21.15.94:0/3104945820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b9415ec00 0x7f6b940b3db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b840097a0 tx=0x7f6b8402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 -- 172.21.15.94:0/3104945820 shutdown_connections 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 --2- 172.21.15.94:0/3104945820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b9415ec00 0x7f6b940b3db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 -- 172.21.15.94:0/3104945820 >> 172.21.15.94:0/3104945820 conn(0x7f6b940d1370 msgr2=0x7f6b940d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:11.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 -- 172.21.15.94:0/3104945820 shutdown_connections 2024-05-27T12:35:11.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.376+0000 7f6b9b200640 1 -- 172.21.15.94:0/3104945820 wait complete. 2024-05-27T12:35:11.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.378+0000 7f6b9b200640 1 Processor -- start 2024-05-27T12:35:11.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.378+0000 7f6b9b200640 1 -- start start 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b940b6370 0x7f6b941f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9415ec00 0x7f6b941f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9415efe0 con 0x7f6b940b3860 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b940b3530 con 0x7f6b9415ec00 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b9b200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9415f160 con 0x7f6b940b6370 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b93e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b93e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37128/0 (socket says 172.21.15.94:37128) 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.379+0000 7f6b93e00640 1 -- 172.21.15.94:0/923384455 learned_addr learned my addr 172.21.15.94:0/923384455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 -- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b940b6370 msgr2=0x7f6b941f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93400640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b940b6370 0x7f6b941f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b99200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9415ec00 0x7f6b941f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b940b6370 0x7f6b941f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 -- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9415ec00 msgr2=0x7f6b941f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9415ec00 0x7f6b941f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 -- 172.21.15.94:0/923384455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b941e87e0 con 0x7f6b940b3860 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b93e00640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b84000c00 tx=0x7f6b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84037040 con 0x7f6b940b3860 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b84042070 con 0x7f6b940b3860 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84030a70 con 0x7f6b940b3860 2024-05-27T12:35:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b941e8ac0 con 0x7f6b940b3860 2024-05-27T12:35:11.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.380+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941e9080 con 0x7f6b940b3860 2024-05-27T12:35:11.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.381+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9406db20 con 0x7f6b940b3860 2024-05-27T12:35:11.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.382+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b84030c10 con 0x7f6b940b3860 2024-05-27T12:35:11.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.383+0000 7f6b8be00640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 0x7f6b7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.383+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b840ac570 con 0x7f6b940b3860 2024-05-27T12:35:11.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.384+0000 7f6b93400640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 0x7f6b7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.385+0000 7f6b93400640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 0x7f6b7406ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007fd0 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.388+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b9406db20 con 0x7f6b940b3860 2024-05-27T12:35:11.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.589+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b94075460 con 0x7f6b940b3860 2024-05-27T12:35:11.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.590+0000 7f6b8be00640 1 -- 172.21.15.94:0/923384455 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f6b840aae20 con 0x7f6b940b3860 2024-05-27T12:35:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:11.590 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 msgr2=0x7f6b7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 0x7f6b7406ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007fd0 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 msgr2=0x7f6b941f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b84000c00 tx=0x7f6b840046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 shutdown_connections 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6b7406c920 0x7f6b7406ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9415ec00 0x7f6b941f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b940b6370 0x7f6b941f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 --2- 172.21.15.94:0/923384455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b940b3860 0x7f6b941f6440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 >> 172.21.15.94:0/923384455 conn(0x7f6b940d1370 msgr2=0x7f6b940cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 shutdown_connections 2024-05-27T12:35:11.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.593+0000 7f6b9b200640 1 -- 172.21.15.94:0/923384455 wait complete. 2024-05-27T12:35:11.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:11.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.819+0000 7f44a5e00640 1 Processor -- start 2024-05-27T12:35:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.819+0000 7f44a5e00640 1 -- start start 2024-05-27T12:35:11.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.820+0000 7f44a5e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f44a015eb70 0x7f44a00d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.820+0000 7f44a5e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00d2df0 0x7f44a00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.820+0000 7f44a5e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a00b82e0 con 0x7f44a00b7e20 2024-05-27T12:35:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.820+0000 7f44a5e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00b8ef0 con 0x7f44a015eb70 2024-05-27T12:35:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.820+0000 7f44a5e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f44a00b85c0 con 0x7f44a00d2df0 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f44a015eb70 0x7f44a00d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f44a015eb70 0x7f44a00d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37132/0 (socket says 172.21.15.94:37132) 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 -- 172.21.15.94:0/4283875284 learned_addr learned my addr 172.21.15.94:0/4283875284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449e000640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00d2df0 0x7f44a00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 -- 172.21.15.94:0/4283875284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00d2df0 msgr2=0x7f44a00b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00d2df0 0x7f44a00b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 -- 172.21.15.94:0/4283875284 >> v1:172.21.15.97:6789/0 conn(0x7f44a00b7e20 legacy=0x7f44a00b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.821+0000 7f449ea00640 1 -- 172.21.15.94:0/4283875284 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a00b88a0 con 0x7f44a015eb70 2024-05-27T12:35:11.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.822+0000 7f449ea00640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a015eb70 0x7f44a00d28b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4490009870 tx=0x7f449002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51af10ae325b943 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.823+0000 7f449d600640 1 -- 172.21.15.94:0/4283875284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490037070 con 0x7f44a015eb70 2024-05-27T12:35:11.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.823+0000 7f449d600640 1 -- 172.21.15.94:0/4283875284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449002ae50 con 0x7f44a015eb70 2024-05-27T12:35:11.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.823+0000 7f449d600640 1 -- 172.21.15.94:0/4283875284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490032860 con 0x7f44a015eb70 2024-05-27T12:35:11.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.823+0000 7f44a5e00640 1 -- 172.21.15.94:0/4283875284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a015eb70 msgr2=0x7f44a00d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.823+0000 7f44a5e00640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a015eb70 0x7f44a00d28b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4490009870 tx=0x7f449002a300 comp rx=0 tx=0).stop 2024-05-27T12:35:11.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 -- 172.21.15.94:0/4283875284 shutdown_connections 2024-05-27T12:35:11.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00d2df0 0x7f44a00b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 --2- 172.21.15.94:0/4283875284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a015eb70 0x7f44a00d28b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 -- 172.21.15.94:0/4283875284 >> 172.21.15.94:0/4283875284 conn(0x7f44a0067a50 msgr2=0x7f44a0067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 -- 172.21.15.94:0/4283875284 shutdown_connections 2024-05-27T12:35:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.824+0000 7f44a5e00640 1 -- 172.21.15.94:0/4283875284 wait complete. 2024-05-27T12:35:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.825+0000 7f44a5e00640 1 Processor -- start 2024-05-27T12:35:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.826+0000 7f44a5e00640 1 -- start start 2024-05-27T12:35:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.826+0000 7f44a5e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00b7e20 0x7f44a014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f44a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a015eb70 0x7f44a00cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f44a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f44a5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44a00b9a90 con 0x7f44a00cd4a0 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f44a5e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44a015ef50 con 0x7f44a015eb70 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f44a5e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f44a00b9c10 con 0x7f44a00b7e20 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37144/0 (socket says 172.21.15.94:37144) 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 -- 172.21.15.94:0/2454349620 learned_addr learned my addr 172.21.15.94:0/2454349620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449e000640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a015eb70 0x7f44a00cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449ea00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00b7e20 0x7f44a014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 -- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00b7e20 msgr2=0x7f44a014ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00b7e20 0x7f44a014ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 -- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a015eb70 msgr2=0x7f44a00cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a015eb70 0x7f44a00cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.827+0000 7f449f400640 1 -- 172.21.15.94:0/2454349620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a014b3c0 con 0x7f44a00cd4a0 2024-05-27T12:35:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f449f400640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f449800c730 tx=0x7f449800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498016020 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498006d50 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a00ce320 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498003bb0 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.828+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a00ce8e0 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.829+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a0075460 con 0x7f44a00cd4a0 2024-05-27T12:35:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.830+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44980050d0 con 0x7f44a00cd4a0 2024-05-27T12:35:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.831+0000 7f4497400640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 0x7f448006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.831+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f449808ba40 con 0x7f44a00cd4a0 2024-05-27T12:35:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.832+0000 7f449ea00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 0x7f448006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:11.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.833+0000 7f449ea00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 0x7f448006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4490034000 tx=0x7f449002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:11.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:11.836+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4498020020 con 0x7f44a00cd4a0 2024-05-27T12:35:12.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.024+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44a00cb1d0 con 0x7f44a00cd4a0 2024-05-27T12:35:12.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.024+0000 7f4497400640 1 -- 172.21.15.94:0/2454349620 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4498059de0 con 0x7f44a00cd4a0 2024-05-27T12:35:12.024 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 msgr2=0x7f448006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 0x7f448006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4490034000 tx=0x7f449002e000 comp rx=0 tx=0).stop 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 msgr2=0x7f44a00cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f449800c730 tx=0x7f449800cbf0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 shutdown_connections 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f448006c8f0 0x7f448006edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44a015eb70 0x7f44a00cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44a00cd4a0 0x7f44a00cd930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 --2- 172.21.15.94:0/2454349620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f44a00b7e20 0x7f44a014ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 >> 172.21.15.94:0/2454349620 conn(0x7f44a0067a50 msgr2=0x7f44a00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 shutdown_connections 2024-05-27T12:35:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.027+0000 7f44a5e00640 1 -- 172.21.15.94:0/2454349620 wait complete. 2024-05-27T12:35:12.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.233+0000 7fe11fe00640 1 Processor -- start 2024-05-27T12:35:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.234+0000 7fe11fe00640 1 -- start start 2024-05-27T12:35:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe118159290 0x7fe1180b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11fe00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe11806fa70 con 0x7fe11806c940 2024-05-27T12:35:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11fe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1181625d0 con 0x7fe1180703e0 2024-05-27T12:35:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe1180b08f0 con 0x7fe118159290 2024-05-27T12:35:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11de00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe11806c940 0x7fe11806f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50348/0 (socket says 172.21.15.94:50348) 2024-05-27T12:35:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11de00640 1 -- 172.21.15.94:0/1977618375 learned_addr learned my addr 172.21.15.94:0/1977618375 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.235+0000 7fe11d400640 1 --2- 172.21.15.94:0/1977618375 >> v2:172.21.15.97:3300/0 conn(0x7fe118159290 0x7fe1180b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854842788 0 0) 0x7fe11806fa70 con 0x7fe11806c940 2024-05-27T12:35:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0fc003520 con 0x7fe11806c940 2024-05-27T12:35:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1263385503 0 0) 0x7fe0fc003520 con 0x7fe11806c940 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 >> v2:172.21.15.97:3300/0 conn(0x7fe118159290 msgr2=0x7fe1180b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 --2- 172.21.15.94:0/1977618375 >> v2:172.21.15.97:3300/0 conn(0x7fe118159290 0x7fe1180b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 >> v1:172.21.15.97:6789/0 conn(0x7fe1180703e0 legacy=0x7fe11806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1181628b0 con 0x7fe11806c940 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fe1100019f0 con 0x7fe11806c940 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 >> v1:172.21.15.94:6789/0 conn(0x7fe11806c940 legacy=0x7fe11806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 0x7fe0fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180b08f0 con 0x7fe0fc0051c0 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc00c060 con 0x7fe0fc004800 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.236+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc00c260 con 0x7fe0fc0039b0 2024-05-27T12:35:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 0x7fe0fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11ca00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11d400640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 -- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0fc0039b0 msgr2=0x7fe0fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0fc0039b0 0x7fe0fc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 -- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc004800 msgr2=0x7fe0fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc004800 0x7fe0fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.237+0000 7fe11de00640 1 -- 172.21.15.94:0/1977618375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc003520 con 0x7fe0fc0051c0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.238+0000 7fe11de00640 1 -- 172.21.15.94:0/1977618375 reap_dead start 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.238+0000 7fe11de00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 0x7fe0fc00b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1180676e0 tx=0x7fe0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c724cfe1698496 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.238+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11000a460 con 0x7fe0fc0051c0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.238+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11002f070 con 0x7fe0fc0051c0 2024-05-27T12:35:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.238+0000 7fe117e00640 1 -- 172.21.15.94:0/1977618375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11002bc50 con 0x7fe0fc0051c0 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 -- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 msgr2=0x7fe0fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 0x7fe0fc00b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1180676e0 tx=0x7fe0fc005830 comp rx=0 tx=0).stop 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 -- 172.21.15.94:0/1977618375 shutdown_connections 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 --2- 172.21.15.94:0/1977618375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0fc0051c0 0x7fe0fc00b9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 -- 172.21.15.94:0/1977618375 >> 172.21.15.94:0/1977618375 conn(0x7fe1180d0d70 msgr2=0x7fe1180d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 -- 172.21.15.94:0/1977618375 shutdown_connections 2024-05-27T12:35:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.239+0000 7fe11fe00640 1 -- 172.21.15.94:0/1977618375 wait complete. 2024-05-27T12:35:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.241+0000 7fe11fe00640 1 Processor -- start 2024-05-27T12:35:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.241+0000 7fe11fe00640 1 -- start start 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc0051c0 0x7fe1181596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 0x7fe1181f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe118163150 con 0x7fe1181f40e0 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe118162e70 con 0x7fe0fc0051c0 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11fe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe118067800 con 0x7fe1180add70 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37166/0 (socket says 172.21.15.94:37166) 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 -- 172.21.15.94:0/234149819 learned_addr learned my addr 172.21.15.94:0/234149819 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11d400640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc0051c0 0x7fe1181596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11ca00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 0x7fe1181f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 -- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 msgr2=0x7fe1181f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 0x7fe1181f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 -- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc0051c0 msgr2=0x7fe1181596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc0051c0 0x7fe1181596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11de00640 1 -- 172.21.15.94:0/234149819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1181512b0 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.242+0000 7fe11ca00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 0x7fe1181f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe11de00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe110000f80 tx=0x7fe1181627a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110044070 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11002bc00 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118151590 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110034470 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.243+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11815a420 con 0x7fe1181f40e0 2024-05-27T12:35:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.244+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe118068a70 con 0x7fe1181f40e0 2024-05-27T12:35:12.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.245+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe110047070 con 0x7fe1181f40e0 2024-05-27T12:35:12.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.246+0000 7fe115600640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.246+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe1100ab750 con 0x7fe1181f40e0 2024-05-27T12:35:12.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.247+0000 7fe11d400640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.248+0000 7fe11d400640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 0x7fe0f806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1181f7920 tx=0x7fe108008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.251+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe110079520 con 0x7fe1181f40e0 2024-05-27T12:35:12.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.448+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe11806d6e0 con 0x7fe1181f40e0 2024-05-27T12:35:12.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.449+0000 7fe115600640 1 -- 172.21.15.94:0/234149819 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe11007d820 con 0x7fe1181f40e0 2024-05-27T12:35:12.449 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 msgr2=0x7fe0f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 0x7fe0f806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe1181f7920 tx=0x7fe108008040 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 msgr2=0x7fe1181579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe110000f80 tx=0x7fe1181627a0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 shutdown_connections 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1181f40e0 0x7fe1181579b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1180add70 0x7fe1181f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 --2- 172.21.15.94:0/234149819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc0051c0 0x7fe1181596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 >> 172.21.15.94:0/234149819 conn(0x7fe1180d0d70 msgr2=0x7fe1180750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:12.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 shutdown_connections 2024-05-27T12:35:12.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.452+0000 7fe11fe00640 1 -- 172.21.15.94:0/234149819 wait complete. 2024-05-27T12:35:12.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:12.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.673+0000 7f8ee9800640 1 Processor -- start 2024-05-27T12:35:12.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.674+0000 7f8ee9800640 1 -- start start 2024-05-27T12:35:12.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee9800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ee40cfb40 0x7f8ee40d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee9800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee40d2180 con 0x7f8ee415eb00 2024-05-27T12:35:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee9800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee415eee0 con 0x7f8ee40d1d70 2024-05-27T12:35:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee9800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee40d5700 con 0x7f8ee40cfb40 2024-05-27T12:35:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee2a00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8ee415eb00 0x7f8ee40d3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50358/0 (socket says 172.21.15.94:50358) 2024-05-27T12:35:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.675+0000 7f8ee2a00640 1 -- 172.21.15.94:0/615373298 learned_addr learned my addr 172.21.15.94:0/615373298 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee2000640 1 --2- 172.21.15.94:0/615373298 >> v2:172.21.15.97:3300/0 conn(0x7f8ee40cfb40 0x7f8ee40d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299316166 0 0) 0x7f8ed4001110 con 0x7f8ee415eb00 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ec8003520 con 0x7f8ee415eb00 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3782162902 0 0) 0x7f8ec8003520 con 0x7f8ee415eb00 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 >> v2:172.21.15.97:3300/0 conn(0x7f8ee40cfb40 msgr2=0x7f8ee40d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 --2- 172.21.15.94:0/615373298 >> v2:172.21.15.97:3300/0 conn(0x7f8ee40cfb40 0x7f8ee40d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 >> v1:172.21.15.97:6789/0 conn(0x7f8ee40d1d70 legacy=0x7f8ee40b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee40d4660 con 0x7f8ee415eb00 2024-05-27T12:35:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f8ed4001840 con 0x7f8ee415eb00 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 >> v1:172.21.15.94:6789/0 conn(0x7f8ee415eb00 legacy=0x7f8ee40d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec8004800 0x7f8ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ec80051c0 0x7f8ec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40d5700 con 0x7f8ec80039b0 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec800c070 con 0x7f8ec80051c0 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.676+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec800c270 con 0x7f8ec8004800 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 -- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec8004800 msgr2=0x7f8ec8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec8004800 0x7f8ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 -- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ec80051c0 msgr2=0x7f8ec800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:12.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ec80051c0 0x7f8ec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee2a00640 1 -- 172.21.15.94:0/615373298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec8003520 con 0x7f8ec80039b0 2024-05-27T12:35:12.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.677+0000 7f8ee3400640 1 -- 172.21.15.94:0/615373298 reap_dead start 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.678+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c4c0 tx=0x7f8ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6f566791684c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.678+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400c0e0 con 0x7f8ec80039b0 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.678+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed4034070 con 0x7f8ec80039b0 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.678+0000 7f8ee1600640 1 -- 172.21.15.94:0/615373298 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4005d70 con 0x7f8ec80039b0 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 -- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 msgr2=0x7f8ec8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c4c0 tx=0x7f8ec8005830 comp rx=0 tx=0).stop 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 -- 172.21.15.94:0/615373298 shutdown_connections 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 --2- 172.21.15.94:0/615373298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 -- 172.21.15.94:0/615373298 >> 172.21.15.94:0/615373298 conn(0x7f8ee4067a50 msgr2=0x7f8ee4067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 -- 172.21.15.94:0/615373298 shutdown_connections 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.679+0000 7f8ee9800640 1 -- 172.21.15.94:0/615373298 wait complete. 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.681+0000 7f8ee9800640 1 Processor -- start 2024-05-27T12:35:12.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.681+0000 7f8ee9800640 1 -- start start 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec80039b0 0x7f8ee414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee415d400 0x7f8ee40c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40d4f00 con 0x7f8ee4161640 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40d4c20 con 0x7f8ee415d400 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee9800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee40d51e0 con 0x7f8ec80039b0 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37182/0 (socket says 172.21.15.94:37182) 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 -- 172.21.15.94:0/1228099224 learned_addr learned my addr 172.21.15.94:0/1228099224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec80039b0 0x7f8ee414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 -- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec80039b0 msgr2=0x7f8ee414d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec80039b0 0x7f8ee414d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.682+0000 7f8ee3400640 1 -- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee415d400 msgr2=0x7f8ee40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee2000640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee415d400 0x7f8ee40c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee3400640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee415d400 0x7f8ee40c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee3400640 1 -- 172.21.15.94:0/1228099224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee40c9c90 con 0x7f8ee4161640 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee3400640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4162500 tx=0x7f8edc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc017070 con 0x7f8ee4161640 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc005d40 con 0x7f8ee4161640 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00a7f0 con 0x7f8ee4161640 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee40c96d0 con 0x7f8ee4161640 2024-05-27T12:35:12.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.683+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee414de00 con 0x7f8ee4161640 2024-05-27T12:35:12.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.684+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee4068a70 con 0x7f8ee4161640 2024-05-27T12:35:12.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.685+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8edc01a070 con 0x7f8ee4161640 2024-05-27T12:35:12.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.686+0000 7f8edaa00640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 0x7f8ec406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:12.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.686+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8edc08ada0 con 0x7f8ee4161640 2024-05-27T12:35:12.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.687+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 0x7f8ec406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:12.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.688+0000 7f8ee2a00640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 0x7f8ec406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c490 tx=0x7f8ed402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:12.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.691+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8edc059260 con 0x7f8ee4161640 2024-05-27T12:35:12.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.887+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ee40b5a10 con 0x7f8ee4161640 2024-05-27T12:35:12.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.888+0000 7f8edaa00640 1 -- 172.21.15.94:0/1228099224 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8edc058b60 con 0x7f8ee4161640 2024-05-27T12:35:12.888 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:12.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 msgr2=0x7f8ec406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 0x7f8ec406ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c490 tx=0x7f8ed402e000 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 msgr2=0x7f8ee4161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4162500 tx=0x7f8edc00cb90 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 shutdown_connections 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ec406c920 0x7f8ec406ede0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee415d400 0x7f8ee40c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee4161640 0x7f8ee4161af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 --2- 172.21.15.94:0/1228099224 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ec80039b0 0x7f8ee414d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 >> 172.21.15.94:0/1228099224 conn(0x7f8ee4067a50 msgr2=0x7f8ee40cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 shutdown_connections 2024-05-27T12:35:12.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:12.891+0000 7f8ee9800640 1 -- 172.21.15.94:0/1228099224 wait complete. 2024-05-27T12:35:12.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:13.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.103+0000 7f0058e00640 1 Processor -- start 2024-05-27T12:35:13.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.103+0000 7f0058e00640 1 -- start start 2024-05-27T12:35:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f005406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 0x7f00540cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00540cf940 0x7f0054071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f00540cfdb0 con 0x7f00540cf940 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0054160f70 con 0x7f005406f280 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.104+0000 7f0058e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0054160360 con 0x7f005415a250 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00540cf940 0x7f0054071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f00540cf940 0x7f0054071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37196/0 (socket says 172.21.15.94:37196) 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 -- 172.21.15.94:0/446325021 learned_addr learned my addr 172.21.15.94:0/446325021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0051e00640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f005406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0049400640 1 --2- 172.21.15.94:0/446325021 >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 0x7f00540cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 -- 172.21.15.94:0/446325021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 msgr2=0x7f005406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f005406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 -- 172.21.15.94:0/446325021 >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 msgr2=0x7f00540cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 --2- 172.21.15.94:0/446325021 >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 0x7f00540cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0052800640 1 -- 172.21.15.94:0/446325021 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0054160640 con 0x7f00540cf940 2024-05-27T12:35:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.105+0000 7f0049400640 1 --2- 172.21.15.94:0/446325021 >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 0x7f00540cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.106+0000 7f0052800640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f0054071f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00540700d0 tx=0x7f004c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e708bedc4755c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.106+0000 7f0051400640 1 -- 172.21.15.94:0/446325021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c036070 con 0x7f00540cf940 2024-05-27T12:35:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.106+0000 7f0051400640 1 -- 172.21.15.94:0/446325021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004c031600 con 0x7f00540cf940 2024-05-27T12:35:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.106+0000 7f0051400640 1 -- 172.21.15.94:0/446325021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c0319d0 con 0x7f00540cf940 2024-05-27T12:35:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 -- 172.21.15.94:0/446325021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 msgr2=0x7f0054071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f0054071f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00540700d0 tx=0x7f004c02a6a0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 -- 172.21.15.94:0/446325021 shutdown_connections 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f0054071f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 --2- 172.21.15.94:0/446325021 >> v2:172.21.15.97:3300/0 conn(0x7f005406f280 0x7f00540cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 --2- 172.21.15.94:0/446325021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f005406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 -- 172.21.15.94:0/446325021 >> 172.21.15.94:0/446325021 conn(0x7f00540b3830 msgr2=0x7f00540b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 -- 172.21.15.94:0/446325021 shutdown_connections 2024-05-27T12:35:13.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.107+0000 7f0058e00640 1 -- 172.21.15.94:0/446325021 wait complete. 2024-05-27T12:35:13.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.109+0000 7f0058e00640 1 Processor -- start 2024-05-27T12:35:13.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.109+0000 7f0058e00640 1 -- start start 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005406f280 0x7f0054145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f00540cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f005415a630 con 0x7f00540cf940 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0054160c00 con 0x7f005406f280 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0058e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f005406e7f0 con 0x7f005415a250 2024-05-27T12:35:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0049400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0049400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37198/0 (socket says 172.21.15.94:37198) 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.110+0000 7f0049400640 1 -- 172.21.15.94:0/1541754668 learned_addr learned my addr 172.21.15.94:0/1541754668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0051e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005406f280 0x7f0054145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0052800640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f00540cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 -- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 msgr2=0x7f00540cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f00540cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 -- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005406f280 msgr2=0x7f0054145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005406f280 0x7f0054145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 -- 172.21.15.94:0/1541754668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0054156970 con 0x7f00540cf940 2024-05-27T12:35:13.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0049400640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c7f0 tx=0x7f003c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c015070 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c007a30 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0054156c50 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.111+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c007e00 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.112+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005415c560 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.113+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0054075460 con 0x7f00540cf940 2024-05-27T12:35:13.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.113+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f003c018070 con 0x7f00540cf940 2024-05-27T12:35:13.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.114+0000 7f004aa00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.114+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f003c08bb70 con 0x7f00540cf940 2024-05-27T12:35:13.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.115+0000 7f0051e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.116+0000 7f0051e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 0x7f003006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0044006fd0 tx=0x7f0044008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.118+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f003c059f90 con 0x7f00540cf940 2024-05-27T12:35:13.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.348+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f005406e1c0 con 0x7f00540cf940 2024-05-27T12:35:13.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.349+0000 7f004aa00640 1 -- 172.21.15.94:0/1541754668 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f003c059890 con 0x7f00540cf940 2024-05-27T12:35:13.349 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:13.349 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:13.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.351+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 msgr2=0x7f003006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.351+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 0x7f003006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0044006fd0 tx=0x7f0044008040 comp rx=0 tx=0).stop 2024-05-27T12:35:13.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.351+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 msgr2=0x7f005415c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.351+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c7f0 tx=0x7f003c00cbf0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 shutdown_connections 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f003006c8f0 0x7f003006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f005415a250 0x7f00540cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00540cf940 0x7f005415c020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 --2- 172.21.15.94:0/1541754668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f005406f280 0x7f0054145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 >> 172.21.15.94:0/1541754668 conn(0x7f00540b3830 msgr2=0x7f005415f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 shutdown_connections 2024-05-27T12:35:13.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.352+0000 7f0058e00640 1 -- 172.21.15.94:0/1541754668 wait complete. 2024-05-27T12:35:13.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.563+0000 7f0877e00640 1 Processor -- start 2024-05-27T12:35:13.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.563+0000 7f0877e00640 1 -- start start 2024-05-27T12:35:13.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0877e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f0878160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0877e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08780ceae0 con 0x7f08781460d0 2024-05-27T12:35:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0877e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08781602c0 con 0x7f0878071d60 2024-05-27T12:35:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0877e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f08781605a0 con 0x7f0878145cf0 2024-05-27T12:35:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0876000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f08781460d0 0x7f08780ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50368/0 (socket says 172.21.15.94:50368) 2024-05-27T12:35:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0876000640 1 -- 172.21.15.94:0/1496365495 learned_addr learned my addr 172.21.15.94:0/1496365495 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0876a00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f0878160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.565+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519036280 0 0) 0x7f08780ceae0 con 0x7f08781460d0 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f085c003520 con 0x7f08781460d0 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1797069231 0 0) 0x7f085c003520 con 0x7f08781460d0 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 msgr2=0x7f0878160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f0878160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 >> v1:172.21.15.97:6789/0 conn(0x7f0878071d60 legacy=0x7f08780cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0878160880 con 0x7f08781460d0 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f0864001670 con 0x7f08781460d0 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 >> v1:172.21.15.94:6789/0 conn(0x7f08781460d0 legacy=0x7f08780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f085c0039e0 0x7f085c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f085c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c004d40 0x7f085c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08781605a0 con 0x7f085c0039e0 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f085c00ba70 con 0x7f085c004d40 2024-05-27T12:35:13.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f085c00bc70 con 0x7f0878145cf0 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.566+0000 7f0876a00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f085c0039e0 0x7f085c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876000640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f085c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876a00640 1 -- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 msgr2=0x7f085c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876a00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f085c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876a00640 1 -- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c004d40 msgr2=0x7f085c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876a00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c004d40 0x7f085c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876a00640 1 -- 172.21.15.94:0/1496365495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c003520 con 0x7f085c0039e0 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.567+0000 7f0876000640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0878145cf0 0x7f085c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0876a00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f085c0039e0 0x7f085c003e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0868009a90 tx=0x7f085c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc067c98ebe55b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868036070 con 0x7f085c0039e0 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868031400 con 0x7f085c0039e0 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0875600640 1 -- 172.21.15.94:0/1496365495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868009690 con 0x7f085c0039e0 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0877e00640 1 -- 172.21.15.94:0/1496365495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f085c0039e0 msgr2=0x7f085c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0877e00640 1 --2- 172.21.15.94:0/1496365495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f085c0039e0 0x7f085c003e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0868009a90 tx=0x7f085c005830 comp rx=0 tx=0).stop 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.568+0000 7f0877400640 1 -- 172.21.15.94:0/1496365495 reap_dead start 2024-05-27T12:35:13.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.569+0000 7f0877e00640 1 -- 172.21.15.94:0/1496365495 shutdown_connections 2024-05-27T12:35:13.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.569+0000 7f0877e00640 1 -- 172.21.15.94:0/1496365495 >> 172.21.15.94:0/1496365495 conn(0x7f08780b2160 msgr2=0x7f08780b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:13.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.569+0000 7f0877e00640 1 -- 172.21.15.94:0/1496365495 shutdown_connections 2024-05-27T12:35:13.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.569+0000 7f0877e00640 1 -- 172.21.15.94:0/1496365495 wait complete. 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.570+0000 7f0877e00640 1 Processor -- start 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.570+0000 7f0877e00640 1 -- start start 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 0x7f0878141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 0x7f08781e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f087806eb00 con 0x7f08781f24a0 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08780cedc0 con 0x7f085c0039e0 2024-05-27T12:35:13.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.571+0000 7f0877e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f087806ec80 con 0x7f08780c7730 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37218/0 (socket says 172.21.15.94:37218) 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 -- 172.21.15.94:0/987401681 learned_addr learned my addr 172.21.15.94:0/987401681 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876a00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 0x7f0878141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0877400640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 0x7f08781e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 -- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 msgr2=0x7f08781e7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 0x7f08781e7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 -- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 msgr2=0x7f0878141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 0x7f0878141030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 -- 172.21.15.94:0/987401681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781f2900 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876a00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 0x7f0878141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0877400640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 0x7f08781e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.572+0000 7f0876000640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f086400d8b0 tx=0x7f086400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.573+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864019070 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.573+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08640074e0 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.573+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640049e0 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.573+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08780c7b90 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.573+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08780c8140 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.574+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08780c8d70 con 0x7f08781f24a0 2024-05-27T12:35:13.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.575+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0864004b80 con 0x7f08781f24a0 2024-05-27T12:35:13.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.576+0000 7f086ea00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 0x7f085806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.576+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f086408b6d0 con 0x7f08781f24a0 2024-05-27T12:35:13.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.577+0000 7f0876a00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 0x7f085806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.578+0000 7f0876a00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 0x7f085806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0868033000 tx=0x7f0868008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.581+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08781f2900 con 0x7f08781f24a0 2024-05-27T12:35:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.768+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0878075460 con 0x7f08781f24a0 2024-05-27T12:35:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.771+0000 7f086ea00640 1 -- 172.21.15.94:0/987401681 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f0864059a60 con 0x7f08781f24a0 2024-05-27T12:35:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 msgr2=0x7f085806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 0x7f085806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0868033000 tx=0x7f0868008030 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 msgr2=0x7f08780b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f086400d8b0 tx=0x7f086400dd70 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 shutdown_connections 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f085806c920 0x7f085806ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08780c7730 0x7f08781e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08781f24a0 0x7f08780b0f60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 --2- 172.21.15.94:0/987401681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f085c0039e0 0x7f0878141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 >> 172.21.15.94:0/987401681 conn(0x7f08780b2160 msgr2=0x7f087815dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 shutdown_connections 2024-05-27T12:35:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.774+0000 7f0877e00640 1 -- 172.21.15.94:0/987401681 wait complete. 2024-05-27T12:35:13.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:13.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.979+0000 7f4e07200640 1 Processor -- start 2024-05-27T12:35:13.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.979+0000 7f4e07200640 1 -- start start 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e07200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162780 0x7f4e080d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e07200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4e08162b60 0x7f4e080d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e07200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0806cc50 con 0x7f4e0815e420 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e07200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4e0806cf30 con 0x7f4e08162b60 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e07200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0806d210 con 0x7f4e08162780 2024-05-27T12:35:13.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e06800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4e0815e420 0x7f4e0815e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50378/0 (socket says 172.21.15.94:50378) 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e06800640 1 -- 172.21.15.94:0/1284228798 learned_addr learned my addr 172.21.15.94:0/1284228798 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4dffe00640 1 --2- 172.21.15.94:0/1284228798 >> v2:172.21.15.94:3300/0 conn(0x7f4e08162b60 0x7f4e080d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e05e00640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162780 0x7f4e080d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.981+0000 7f4e05400640 1 -- 172.21.15.94:0/1284228798 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760468244 0 0) 0x7f4e0806cc50 con 0x7f4e0815e420 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.982+0000 7f4e05400640 1 -- 172.21.15.94:0/1284228798 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dec003520 con 0x7f4e0815e420 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.982+0000 7f4dffe00640 1 -- 172.21.15.94:0/1284228798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162780 msgr2=0x7f4e080d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.982+0000 7f4dffe00640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162780 0x7f4e080d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.982+0000 7f4dffe00640 1 -- 172.21.15.94:0/1284228798 >> v1:172.21.15.94:6789/0 conn(0x7f4e0815e420 legacy=0x7f4e0815e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.982+0000 7f4dffe00640 1 -- 172.21.15.94:0/1284228798 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0806d4f0 con 0x7f4e08162b60 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4dffe00640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e08162b60 0x7f4e080d1790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccffc236228d1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4e05400640 1 -- 172.21.15.94:0/1284228798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0037070 con 0x7f4e08162b60 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4e05400640 1 -- 172.21.15.94:0/1284228798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df002ae50 con 0x7f4e08162b60 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4e05400640 1 -- 172.21.15.94:0/1284228798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0032660 con 0x7f4e08162b60 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4e07200640 1 -- 172.21.15.94:0/1284228798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e08162b60 msgr2=0x7f4e080d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.983+0000 7f4e07200640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e08162b60 0x7f4e080d1790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097a0 tx=0x7f4df002a100 comp rx=0 tx=0).stop 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 -- 172.21.15.94:0/1284228798 shutdown_connections 2024-05-27T12:35:13.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e08162b60 0x7f4e080d1790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 --2- 172.21.15.94:0/1284228798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162780 0x7f4e080d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 -- 172.21.15.94:0/1284228798 >> 172.21.15.94:0/1284228798 conn(0x7f4e080b4070 msgr2=0x7f4e080b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:13.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 -- 172.21.15.94:0/1284228798 shutdown_connections 2024-05-27T12:35:13.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.984+0000 7f4e07200640 1 -- 172.21.15.94:0/1284228798 wait complete. 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.985+0000 7f4e07200640 1 Processor -- start 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.985+0000 7f4e07200640 1 -- start start 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 0x7f4e080c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162b60 0x7f4e0815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08168a20 con 0x7f4e0815e420 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e081688a0 con 0x7f4e08162780 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.986+0000 7f4e07200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4e08168ba0 con 0x7f4e08162b60 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37240/0 (socket says 172.21.15.94:37240) 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 -- 172.21.15.94:0/801594870 learned_addr learned my addr 172.21.15.94:0/801594870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4dffe00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 0x7f4e080c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e06800640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162b60 0x7f4e0815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 -- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162b60 msgr2=0x7f4e0815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162b60 0x7f4e0815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 -- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 msgr2=0x7f4e080c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 0x7f4e080c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 -- 172.21.15.94:0/801594870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e080c9b90 con 0x7f4e0815e420 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4dffe00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 0x7f4e080c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.987+0000 7f4e05e00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df800d5f0 tx=0x7f4df800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.988+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df80125f0 con 0x7f4e0815e420 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.988+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8019070 con 0x7f4e0815e420 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.988+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0815fb50 con 0x7f4e0815e420 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.988+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df80129d0 con 0x7f4e0815e420 2024-05-27T12:35:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.988+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e080cb600 con 0x7f4e0815e420 2024-05-27T12:35:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.989+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e08075460 con 0x7f4e0815e420 2024-05-27T12:35:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.990+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4df8012b70 con 0x7f4e0815e420 2024-05-27T12:35:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.991+0000 7f4dfe000640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.991+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4df808bed0 con 0x7f4e0815e420 2024-05-27T12:35:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.991+0000 7f4dffe00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.992+0000 7f4dffe00640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 0x7f4de406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4df0034000 tx=0x7f4df002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:13.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:13.995+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4df805a390 con 0x7f4e0815e420 2024-05-27T12:35:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.216+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e0806fc20 con 0x7f4e0815e420 2024-05-27T12:35:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.218+0000 7f4dfe000640 1 -- 172.21.15.94:0/801594870 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f4df8059c90 con 0x7f4e0815e420 2024-05-27T12:35:14.217 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:14.217 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 msgr2=0x7f4de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 0x7f4de406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4df0034000 tx=0x7f4df002e000 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 msgr2=0x7f4e08159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df800d5f0 tx=0x7f4df800dab0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 shutdown_connections 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4de406c8f0 0x7f4de406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e08162780 0x7f4e080c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e08162b60 0x7f4e0815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 --2- 172.21.15.94:0/801594870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e0815e420 0x7f4e08159a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 >> 172.21.15.94:0/801594870 conn(0x7f4e080b4070 msgr2=0x7f4e080d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 shutdown_connections 2024-05-27T12:35:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.220+0000 7f4e07200640 1 -- 172.21.15.94:0/801594870 wait complete. 2024-05-27T12:35:14.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T12:34:59.347690+0000', 'modified': '2024-05-27T12:35:02.434158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4244': {'gid': 4244, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6836/1775076322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1775076322}, {'type': 'v1', 'addr': '172.21.15.97:6836', 'nonce': 1775076322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4244, 'qdb_cluster': [4244]} 2024-05-27T12:35:14.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T12:35:14.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-27T12:35:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.851+0000 7fdcd1200640 1 Processor -- start 2024-05-27T12:35:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.851+0000 7fdcd1200640 1 -- start start 2024-05-27T12:35:14.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.852+0000 7fdcd1200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 0x7fdccc0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.852+0000 7fdcd1200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdccc0cd8a0 0x7fdccc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcd1200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdccc15cfd0 con 0x7fdccc0cdd00 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcd1200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdccc15dbe0 con 0x7fdccc0cd8a0 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcd1200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc15d2b0 con 0x7fdccc146060 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcca800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fdccc0cdd00 0x7fdccc072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50392/0 (socket says 172.21.15.94:50392) 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcca800640 1 -- 172.21.15.94:0/3247620759 learned_addr learned my addr 172.21.15.94:0/3247620759 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 0x7fdccc0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/3247620759 >> v2:172.21.15.97:3300/0 conn(0x7fdccc0cd8a0 0x7fdccc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1812313080 0 0) 0x7fdccc15cfd0 con 0x7fdccc0cdd00 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.853+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcb0003520 con 0x7fdccc0cdd00 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3708459807 0 0) 0x7fdcb0003520 con 0x7fdccc0cdd00 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 msgr2=0x7fdccc0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 0x7fdccc0b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 >> v2:172.21.15.97:3300/0 conn(0x7fdccc0cd8a0 msgr2=0x7fdccc06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 --2- 172.21.15.94:0/3247620759 >> v2:172.21.15.97:3300/0 conn(0x7fdccc0cd8a0 0x7fdccc06b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc15d590 con 0x7fdccc0cdd00 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fdcc40019f0 con 0x7fdccc0cdd00 2024-05-27T12:35:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 >> v1:172.21.15.94:6789/0 conn(0x7fdccc0cdd00 legacy=0x7fdccc072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 0x7fdcb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcb0004d40 0x7fdcb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc15d2b0 con 0x7fdcb00039b0 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb000b9c0 con 0x7fdcb0004d40 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb000bb40 con 0x7fdccc146060 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 -- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 msgr2=0x7fdcb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc146060 0x7fdcb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 -- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcb0004d40 msgr2=0x7fdcb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcb0004d40 0x7fdcb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.854+0000 7fdcc9e00640 1 -- 172.21.15.94:0/3247620759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb0003520 con 0x7fdcb00039b0 2024-05-27T12:35:14.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.855+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ac0 tx=0x7fdcb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c65d37110a3fb28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc036070 con 0x7fdcb00039b0 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc031400 con 0x7fdcb00039b0 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcc9400640 1 -- 172.21.15.94:0/3247620759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc009690 con 0x7fdcb00039b0 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 -- 172.21.15.94:0/3247620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 msgr2=0x7fdcb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 --2- 172.21.15.94:0/3247620759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ac0 tx=0x7fdcb0005830 comp rx=0 tx=0).stop 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcca800640 1 -- 172.21.15.94:0/3247620759 reap_dead start 2024-05-27T12:35:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 -- 172.21.15.94:0/3247620759 shutdown_connections 2024-05-27T12:35:14.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 -- 172.21.15.94:0/3247620759 >> 172.21.15.94:0/3247620759 conn(0x7fdccc0b2240 msgr2=0x7fdccc0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 -- 172.21.15.94:0/3247620759 shutdown_connections 2024-05-27T12:35:14.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.856+0000 7fdcd1200640 1 -- 172.21.15.94:0/3247620759 wait complete. 2024-05-27T12:35:14.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.858+0000 7fdcd1200640 1 Processor -- start 2024-05-27T12:35:14.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.858+0000 7fdcd1200640 1 -- start start 2024-05-27T12:35:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc1577f0 0x7fdccc157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 0x7fdccc159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0d0110 con 0x7fdcb00039b0 2024-05-27T12:35:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc146440 con 0x7fdccc159660 2024-05-27T12:35:14.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.859+0000 7fdcd1200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc0b3f70 con 0x7fdccc1577f0 2024-05-27T12:35:14.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37250/0 (socket says 172.21.15.94:37250) 2024-05-27T12:35:14.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1465090883 learned_addr learned my addr 172.21.15.94:0/1465090883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:14.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc1577f0 0x7fdccc157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcca800640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 0x7fdccc159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc1577f0 msgr2=0x7fdccc157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc1577f0 0x7fdccc157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 msgr2=0x7fdccc159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 0x7fdccc159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1465090883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc140be0 con 0x7fdcb00039b0 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcca800640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 0x7fdccc159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.860+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009be0 tx=0x7fdcbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.861+0000 7fdcc2000640 1 -- 172.21.15.94:0/1465090883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc03f020 con 0x7fdcb00039b0 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.861+0000 7fdcc2000640 1 -- 172.21.15.94:0/1465090883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc037040 con 0x7fdcb00039b0 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.861+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc15a490 con 0x7fdcb00039b0 2024-05-27T12:35:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.861+0000 7fdcc2000640 1 -- 172.21.15.94:0/1465090883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc036070 con 0x7fdcb00039b0 2024-05-27T12:35:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.861+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc15aa50 con 0x7fdcb00039b0 2024-05-27T12:35:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.863+0000 7fdcc2000640 1 -- 172.21.15.94:0/1465090883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcbc042020 con 0x7fdcb00039b0 2024-05-27T12:35:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.864+0000 7fdcc2000640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 0x7fdca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.865+0000 7fdcc2000640 1 -- 172.21.15.94:0/1465090883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcbc0abb50 con 0x7fdcb00039b0 2024-05-27T12:35:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.865+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 0x7fdca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.865+0000 7fdcd3f83740 1 Processor -- start 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 -- start start 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 0x7fdca806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0b5750 tx=0x7fdcb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 0x564613383e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x5646133a07f0 0x5646133a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646132b9b10 con 0x564613384370 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646132602b0 con 0x5646133806b0 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.866+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x5646132b5ad0 con 0x5646133a07f0 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37254/0 (socket says 172.21.15.94:37254) 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 -- 172.21.15.94:0/167467176 learned_addr learned my addr 172.21.15.94:0/167467176 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 0x564613383e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcca800640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x5646133a07f0 0x5646133a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 -- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x5646133a07f0 msgr2=0x5646133a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x5646133a07f0 0x5646133a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 -- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 msgr2=0x564613383e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 0x564613383e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 -- 172.21.15.94:0/167467176 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564612fcc2d0 con 0x564613384370 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 0x564613383e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.867+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011c20 tx=0x7fdcb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f00664e0a7df53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcbaa00640 1 -- 172.21.15.94:0/167467176 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4026070 con 0x564613384370 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcbaa00640 1 -- 172.21.15.94:0/167467176 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb401a030 con 0x564613384370 2024-05-27T12:35:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcbaa00640 1 -- 172.21.15.94:0/167467176 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4015e00 con 0x564613384370 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 -- 172.21.15.94:0/167467176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 msgr2=0x5646133a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011c20 tx=0x7fdcb4014000 comp rx=0 tx=0).stop 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 -- 172.21.15.94:0/167467176 shutdown_connections 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x5646133a07f0 0x5646133a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133a02b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/167467176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133806b0 0x564613383e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.868+0000 7fdcd3f83740 1 -- 172.21.15.94:0/167467176 >> 172.21.15.94:0/167467176 conn(0x564613356690 msgr2=0x564613357690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.869+0000 7fdcd3f83740 1 -- 172.21.15.94:0/167467176 shutdown_connections 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.869+0000 7fdcd3f83740 1 -- 172.21.15.94:0/167467176 wait complete. 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.869+0000 7fdcd3f83740 1 -- start start 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.869+0000 7fdcd3f83740 1 Processor -- start 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613398630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x5646133aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646133544c0 con 0x564613384370 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564613273570 con 0x5646133a07f0 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.870+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x5646133547a0 con 0x56461337c330 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcd3f83740 10 client.0 _ll_register_callbacks cb 0x564613241fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37264/0 (socket says 172.21.15.94:37264) 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1408111796 learned_addr learned my addr 172.21.15.94:0/1408111796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 msgr2=0x5646133aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcca800640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x5646133aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x5646133aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 msgr2=0x564613398630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613398630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613398630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 -- 172.21.15.94:0/1408111796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646130966b0 con 0x564613384370 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009bb0 tx=0x7fdcbc079930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af5d9c7999d12f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcaf400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.871+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc077b70 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcbc081fc0 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb401a030 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc0821c0 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 10 client.4569 fetch_fsmap learned FSMap version 8 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5646133af830 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb401a030 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fdcbc07aba0 con 0x564613384370 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 10 client.4569 fetch_fsmap finished waiting for FSMap version 8 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 10 client.4569 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 20 client.4569 populate_metadata read hostname 'smithi094' 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x5646133963e0 0x5646133967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 4 client.4569 mds_command: new command op to 4244 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.872+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 --> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646132c7530 con 0x5646133963e0 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.873+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x5646133963e0 0x5646133967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.874+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x5646133963e0 0x5646133967c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36222e43a38ea4d6 server_cookie=1079526a583ea4d6 in_seq=0 out_seq=0 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.874+0000 7fdcaf400640 10 client.4569 ms_handle_connect on v2:172.21.15.97:6834/1775076322 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.875+0000 7fdcaf400640 1 -- 172.21.15.94:0/1408111796 <== mds.0 v2:172.21.15.97:6834/1775076322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646132c7530 con 0x5646133963e0 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.875+0000 7fdcaf400640 10 client.4569 handle_command_reply: tid=0 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.875+0000 7fdcd3f83740 1 client.4569 shutdown 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 msgr2=0x5646133865e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009bb0 tx=0x7fdcbc079930 comp rx=0 tx=0).stop 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 shutdown_connections 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x5646133963e0 0x5646133967c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613398630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x5646133aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/1408111796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564613384370 0x5646133865e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 >> 172.21.15.94:0/1408111796 conn(0x5646133806b0 msgr2=0x56461335a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcaf400640 0 client.4569 ms_handle_reset on v2:172.21.15.97:6834/1775076322 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 shutdown_connections 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 1 -- 172.21.15.94:0/1408111796 wait complete. 2024-05-27T12:35:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.876+0000 7fdcd3f83740 20 client.4569 trim_cache size 0 max 16384 2024-05-27T12:35:14.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.896+0000 7fdcd3f83740 1 Processor -- start 2024-05-27T12:35:14.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.896+0000 7fdcd3f83740 1 -- start start 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x564613385af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564613384370 0x56461339c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc06e7c0 con 0x5646133a07f0 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc130a10 con 0x564613384370 2024-05-27T12:35:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc06e170 con 0x56461337c330 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37276/0 (socket says 172.21.15.94:37276) 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 -- 172.21.15.94:0/3751983967 learned_addr learned my addr 172.21.15.94:0/3751983967 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x564613385af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564613384370 0x56461339c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 -- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 msgr2=0x564613385af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x564613385af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 -- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564613384370 msgr2=0x56461339c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564613384370 0x56461339c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.897+0000 7fdcca800640 1 -- 172.21.15.94:0/3751983967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461325f8b0 con 0x5646133a07f0 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcca800640 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0b37f0 tx=0x7fdcc4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d924082834b25dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcba000640 1 -- 172.21.15.94:0/3751983967 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400e950 con 0x5646133a07f0 2024-05-27T12:35:14.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcba000640 1 -- 172.21.15.94:0/3751983967 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc400eaf0 con 0x5646133a07f0 2024-05-27T12:35:14.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcba000640 1 -- 172.21.15.94:0/3751983967 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4007110 con 0x5646133a07f0 2024-05-27T12:35:14.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 -- 172.21.15.94:0/3751983967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 msgr2=0x56461339c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0b37f0 tx=0x7fdcc4007f60 comp rx=0 tx=0).stop 2024-05-27T12:35:14.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 -- 172.21.15.94:0/3751983967 shutdown_connections 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564613384370 0x56461339c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646133a07f0 0x56461339c600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/3751983967 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x56461337c330 0x564613385af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.898+0000 7fdcd3f83740 1 -- 172.21.15.94:0/3751983967 >> 172.21.15.94:0/3751983967 conn(0x5646133806b0 msgr2=0x5646133865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.899+0000 7fdcd3f83740 1 -- 172.21.15.94:0/3751983967 shutdown_connections 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.899+0000 7fdcd3f83740 1 -- 172.21.15.94:0/3751983967 wait complete. 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.899+0000 7fdcd3f83740 1 -- start start 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.900+0000 7fdcd3f83740 1 Processor -- start 2024-05-27T12:35:14.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.900+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.900+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x564613384370 0x564613394990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.900+0000 7fdcd3f83740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613418710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646133544c0 con 0x56461337c330 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564613273570 con 0x5646133a07f0 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcd3f83740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x564613398b70 con 0x564613384370 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcd3f83740 10 client.0 _ll_register_callbacks cb 0x7fdcbc075b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37280/0 (socket says 172.21.15.94:37280) 2024-05-27T12:35:14.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 -- 172.21.15.94:0/4126505984 learned_addr learned my addr 172.21.15.94:0/4126505984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x564613384370 0x564613394990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 -- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x564613384370 msgr2=0x564613394990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcca800640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613418710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x564613384370 0x564613394990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 -- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 msgr2=0x564613418710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613418710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 -- 172.21.15.94:0/4126505984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461339efa0 con 0x56461337c330 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcc9e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc081f20 tx=0x7fdcbc046070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d821dfc1d0c54d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.901+0000 7fdcaf400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc077b70 con 0x56461337c330 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcbc081fc0 con 0x56461337c330 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc400eaf0 con 0x56461337c330 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc079f70 con 0x56461337c330 2024-05-27T12:35:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 10 client.4575 fetch_fsmap learned FSMap version 8 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x564613394fd0 con 0x56461337c330 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc400eaf0 con 0x56461337c330 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fdcbc078d10 con 0x56461337c330 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 10 client.4575 fetch_fsmap finished waiting for FSMap version 8 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 10 client.4575 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 20 client.4575 populate_metadata read hostname 'smithi094' 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x56461339c790 0x5646133a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 4 client.4575 mds_command: new command op to 4244 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.902+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 --> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] -- command(tid 0: {"prefix": "status"}) -- 0x5646132c13a0 con 0x56461339c790 2024-05-27T12:35:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.903+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x56461339c790 0x5646133a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:14.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcc3e00640 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x56461339c790 0x5646133a23b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ee0eb5885a7c7b server_cookie=51266233995a1739 in_seq=0 out_seq=0 2024-05-27T12:35:14.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcaf400640 10 client.4575 ms_handle_connect on v2:172.21.15.97:6834/1775076322 2024-05-27T12:35:14.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcaf400640 1 -- 172.21.15.94:0/4126505984 <== mds.0 v2:172.21.15.97:6834/1775076322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5646132c13a0 con 0x56461339c790 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcaf400640 10 client.4575 handle_command_reply: tid=0 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 client.4575 shutdown 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 msgr2=0x56461334f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc081f20 tx=0x7fdcbc046070 comp rx=0 tx=0).stop 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 shutdown_connections 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:6834/1775076322,v1:172.21.15.97:6836/1775076322] conn(0x56461339c790 0x5646133a23b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646133a07f0 0x564613418710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x564613384370 0x564613394990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 --2- 172.21.15.94:0/4126505984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56461337c330 0x56461334f8b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcaf400640 0 client.4575 ms_handle_reset on v2:172.21.15.97:6834/1775076322 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 >> 172.21.15.94:0/4126505984 conn(0x5646133806b0 msgr2=0x564613396340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.904+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 shutdown_connections 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd3f83740 1 -- 172.21.15.94:0/4126505984 wait complete. 2024-05-27T12:35:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd3f83740 20 client.4575 trim_cache size 0 max 16384 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4244, 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 14.96865249, 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 8, 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 20, 2024-05-27T12:35:14.909 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 20, 2024-05-27T12:35:14.910 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 22.964220699999998 2024-05-27T12:35:14.910 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:35:14.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 msgr2=0x7fdca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 0x7fdca806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0b5750 tx=0x7fdcb4008040 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 msgr2=0x7fdccc1f2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009be0 tx=0x7fdcbc004730 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 shutdown_connections 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fdca806c8f0 0x7fdca806edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdccc159660 0x7fdccc159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdccc1577f0 0x7fdccc157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 --2- 172.21.15.94:0/1465090883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcb00039b0 0x7fdccc1f2dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 >> 172.21.15.94:0/1465090883 conn(0x7fdccc0b2240 msgr2=0x7fdccc15e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.905+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 shutdown_connections 2024-05-27T12:35:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:14.906+0000 7fdcd1200640 1 -- 172.21.15.94:0/1465090883 wait complete. 2024-05-27T12:35:14.924 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T12:35:14.935 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T12:35:14.935 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T12:35:14.936 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T12:35:14.936 INFO:teuthology.run_tasks:Running task kclient... 2024-05-27T12:35:14.967 INFO:tasks.kclient:Mounting kernel clients... 2024-05-27T12:35:14.967 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-27T12:35:14.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:14.968 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-05-27T12:35:15.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-05-27T12:35:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-05-27T12:35:15.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:35:15.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.104 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-05-27T12:35:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-05-27T12:35:15.280 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-05-27T12:35:15.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:35:15.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.285 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-05-27T12:35:15.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.345 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-05-27T12:35:15.434 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-05-27T12:35:15.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:35:15.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.438 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-05-27T12:35:15.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:15.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-05-27T12:35:15.565 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-05-27T12:35:15.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T12:35:15.569 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-27T12:35:15.569 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T12:35:15.569 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T12:35:15.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T12:35:15.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.773+0000 7f7227400640 1 Processor -- start 2024-05-27T12:35:15.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.773+0000 7f7227400640 1 -- start start 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.774+0000 7f7227400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f722015ec00 0x7f72200b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.774+0000 7f7227400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b6370 0x7f72200b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f7227400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72201632f0 con 0x7f72200b3860 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f7227400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f72200cf2e0 con 0x7f722015ec00 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f7227400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f72201635d0 con 0x7f72200b6370 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f721fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f72200b3860 0x7f72200cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:50408/0 (socket says 172.21.15.94:50408) 2024-05-27T12:35:15.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f721fe00640 1 -- 172.21.15.94:0/3667112849 learned_addr learned my addr 172.21.15.94:0/3667112849 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:15.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f7224a00640 1 --2- 172.21.15.94:0/3667112849 >> v2:172.21.15.97:3300/0 conn(0x7f722015ec00 0x7f72200b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:15.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.775+0000 7f7225400640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b6370 0x7f72200b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:15.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.776+0000 7f7224a00640 1 -- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b6370 msgr2=0x7f72200b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:15.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.776+0000 7f7224a00640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b6370 0x7f72200b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:15.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.776+0000 7f7224a00640 1 -- 172.21.15.94:0/3667112849 >> v1:172.21.15.94:6789/0 conn(0x7f72200b3860 legacy=0x7f72200cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:15.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.776+0000 7f721f400640 1 -- 172.21.15.94:0/3667112849 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254228820 0 0) 0x7f72201632f0 con 0x7f72200b3860 2024-05-27T12:35:15.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.776+0000 7f7224a00640 1 -- 172.21.15.94:0/3667112849 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72200cf530 con 0x7f722015ec00 2024-05-27T12:35:15.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.777+0000 7f7224a00640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f722015ec00 0x7f72200b3db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72140097a0 tx=0x7f721402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5b5c2ae8967965 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:15.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.778+0000 7f721f400640 1 -- 172.21.15.94:0/3667112849 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214037070 con 0x7f722015ec00 2024-05-27T12:35:15.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.778+0000 7f721f400640 1 -- 172.21.15.94:0/3667112849 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7214032440 con 0x7f722015ec00 2024-05-27T12:35:15.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.778+0000 7f721f400640 1 -- 172.21.15.94:0/3667112849 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214032810 con 0x7f722015ec00 2024-05-27T12:35:15.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 -- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f722015ec00 msgr2=0x7f72200b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:15.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f722015ec00 0x7f72200b3db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72140097a0 tx=0x7f721402a180 comp rx=0 tx=0).stop 2024-05-27T12:35:15.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 -- 172.21.15.94:0/3667112849 shutdown_connections 2024-05-27T12:35:15.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b6370 0x7f72200b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:15.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 --2- 172.21.15.94:0/3667112849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f722015ec00 0x7f72200b3db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:15.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 -- 172.21.15.94:0/3667112849 >> 172.21.15.94:0/3667112849 conn(0x7f72200d1370 msgr2=0x7f72200d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:15.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 -- 172.21.15.94:0/3667112849 shutdown_connections 2024-05-27T12:35:15.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.779+0000 7f7227400640 1 -- 172.21.15.94:0/3667112849 wait complete. 2024-05-27T12:35:15.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.781+0000 7f7227400640 1 Processor -- start 2024-05-27T12:35:15.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.781+0000 7f7227400640 1 -- start start 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b3860 0x7f72200cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72200b6370 0x7f7220152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f722015efe0 con 0x7f722015ec00 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72200b3530 con 0x7f72200b6370 2024-05-27T12:35:15.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7227400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f722015f160 con 0x7f72200b3860 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7225400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7225400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37284/0 (socket says 172.21.15.94:37284) 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f7225400640 1 -- 172.21.15.94:0/988081711 learned_addr learned my addr 172.21.15.94:0/988081711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.782+0000 7f721fe00640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72200b6370 0x7f7220152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 -- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b3860 msgr2=0x7f72200cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b3860 0x7f72200cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 -- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72200b6370 msgr2=0x7f7220152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72200b6370 0x7f7220152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 -- 172.21.15.94:0/988081711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72200ce770 con 0x7f722015ec00 2024-05-27T12:35:15.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7225400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7218009dd0 tx=0x7f721800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218018020 con 0x7f722015ec00 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7218014480 con 0x7f722015ec00 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218014870 con 0x7f722015ec00 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72200cded0 con 0x7f722015ec00 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.783+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72200ce490 con 0x7f722015ec00 2024-05-27T12:35:15.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.784+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722006db20 con 0x7f722015ec00 2024-05-27T12:35:15.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.785+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7218005d10 con 0x7f722015ec00 2024-05-27T12:35:15.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.786+0000 7f721cc00640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 0x7f720006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:15.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.786+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f721808c5c0 con 0x7f722015ec00 2024-05-27T12:35:15.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.787+0000 7f7224a00640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 0x7f720006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:15.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.788+0000 7f7224a00640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 0x7f720006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7214034000 tx=0x7f721402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:15.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:15.791+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f722006db20 con 0x7f722015ec00 2024-05-27T12:35:16.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.042+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7220075460 con 0x7f722015ec00 2024-05-27T12:35:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.043+0000 7f721cc00640 1 -- 172.21.15.94:0/988081711 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f7218022070 con 0x7f722015ec00 2024-05-27T12:35:16.042 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T12:35:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 msgr2=0x7f720006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 0x7f720006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7214034000 tx=0x7f721402c000 comp rx=0 tx=0).stop 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 msgr2=0x7f72201606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7218009dd0 tx=0x7f721800c490 comp rx=0 tx=0).stop 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 shutdown_connections 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f720006c8f0 0x7f720006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72200b6370 0x7f7220152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f722015ec00 0x7f72201606d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 --2- 172.21.15.94:0/988081711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72200b3860 0x7f72200cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:16.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.045+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 >> 172.21.15.94:0/988081711 conn(0x7f72200d1370 msgr2=0x7f72200b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:16.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.046+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 shutdown_connections 2024-05-27T12:35:16.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16.046+0000 7f7227400640 1 -- 172.21.15.94:0/988081711 wait complete. 2024-05-27T12:35:16.063 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T12:35:16.063 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T12:35:16.063 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T12:35:16.064 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T12:35:16.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:16.064 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-05-27T12:35:16.110 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-05-27T12:35:16.111 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-05-27T12:35:16.112 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:16.113 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T12:35:16.113 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:16.113 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:16.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-05-27T12:35:16.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-05-27T12:35:16.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-05-27T12:35:16.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T12:35:16.114 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:16.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:16.604 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T12:35:16.676 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-05-27T12:35:16.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:16.680 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-05-27T12:35:16.734 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-05-27T12:35:16.734 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T12:35:16.734 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T12:35:16.734 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T12:35:16.734 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T12:35:16.734 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:16.735 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:16.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T12:35:16.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T12:35:16.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T12:35:16.735 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:16.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:17.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:17.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:17.443 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-05-27T12:35:17.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:17.499 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-05-27T12:35:17.554 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:17.554 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:17.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T12:35:17.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T12:35:17.554 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:17.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:17.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:17.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T12:35:17.742 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:17.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:18.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:18.036 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:18.036 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:18.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-05-27T12:35:18.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T12:35:18.037 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:18.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:18.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:18.236 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T12:35:18.279 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T12:35:18.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:18.279 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T12:35:18.371 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-27T12:35:18.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:18.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:18.492 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:18.492 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:19.240 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:19.240 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:19.240 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-05-27T12:35:19.240 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 0@b6a8d677-9e12-4245-aa4c-e7c6ae09d474.cephfs=/ 2024-05-27T12:35:19.240 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.94:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2024-05-27T12:35:19.243 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T12:35:19.243 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-05-27T12:35:19.243 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T12:35:19.243 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T12:35:19.243 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T12:35:19.247 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T12:35:19.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:19.247 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T12:35:19.328 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:19.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:19.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:19.470 INFO:teuthology.orchestra.run.smithi094.stdout:client4501 2024-05-27T12:35:19.470 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:19.470 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:19.523 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:19.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:19.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:19.667 INFO:teuthology.orchestra.run.smithi094.stdout:client4501 2024-05-27T12:35:19.667 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:19.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:19.809 INFO:teuthology.orchestra.run.smithi094.stdout:client4501 2024-05-27T12:35:19.809 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b6a8d677-9e12-4245-aa4c-e7c6ae09d474.client4501/status') 2024-05-27T12:35:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4501 (3)172.21.15.94:0/233466774 2024-05-27T12:35:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-05-27T12:35:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-05-27T12:35:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-05-27T12:35:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:69 bytes copied, 8.0709e-05 s, 855 kB/s 2024-05-27T12:35:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:19.966 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-27T12:35:19.966 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T12:35:19.966 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-27T12:35:19.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T12:35:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.164+0000 7f0bc9600640 1 Processor -- start 2024-05-27T12:35:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.164+0000 7f0bc9600640 1 -- start start 2024-05-27T12:35:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bc9600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0bc415e920 0x7f0bc4162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bc9600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc415ed00 con 0x7f0bc40b2f10 2024-05-27T12:35:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bc9600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc40cd580 con 0x7f0bc4072630 2024-05-27T12:35:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bc9600640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc40cd860 con 0x7f0bc415e920 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bc2a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0bc415e920 0x7f0bc4162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bba000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0bc40b2f10 0x7f0bc4071f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:37778/0 (socket says 172.21.15.94:37778) 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.166+0000 7f0bba000640 1 -- 172.21.15.94:0/1572733220 learned_addr learned my addr 172.21.15.94:0/1572733220 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.167+0000 7f0bc2a00640 1 -- 172.21.15.94:0/1572733220 >> v1:172.21.15.97:6789/0 conn(0x7f0bc4072630 legacy=0x7f0bc40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.167+0000 7f0bc2a00640 1 -- 172.21.15.94:0/1572733220 >> v1:172.21.15.94:6789/0 conn(0x7f0bc40b2f10 legacy=0x7f0bc4071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.167+0000 7f0bc2a00640 1 -- 172.21.15.94:0/1572733220 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40cdb40 con 0x7f0bc415e920 2024-05-27T12:35:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.167+0000 7f0bc2000640 1 -- 172.21.15.94:0/1572733220 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3327408042 0 0) 0x7f0bc415ed00 con 0x7f0bc40b2f10 2024-05-27T12:35:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.168+0000 7f0bc2a00640 1 --2- 172.21.15.94:0/1572733220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc4162e40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40097a0 tx=0x7f0bb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef186c391c743add server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.168+0000 7f0bc2000640 1 -- 172.21.15.94:0/1572733220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4037070 con 0x7f0bc415e920 2024-05-27T12:35:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.168+0000 7f0bc2000640 1 -- 172.21.15.94:0/1572733220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb4032440 con 0x7f0bc415e920 2024-05-27T12:35:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.168+0000 7f0bc2000640 1 -- 172.21.15.94:0/1572733220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4032810 con 0x7f0bc415e920 2024-05-27T12:35:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 -- 172.21.15.94:0/1572733220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 msgr2=0x7f0bc4162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1572733220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc4162e40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40097a0 tx=0x7f0bb402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 -- 172.21.15.94:0/1572733220 shutdown_connections 2024-05-27T12:35:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1572733220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc4162e40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 -- 172.21.15.94:0/1572733220 >> 172.21.15.94:0/1572733220 conn(0x7f0bc40b3830 msgr2=0x7f0bc40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 -- 172.21.15.94:0/1572733220 shutdown_connections 2024-05-27T12:35:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.169+0000 7f0bc9600640 1 -- 172.21.15.94:0/1572733220 wait complete. 2024-05-27T12:35:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.171+0000 7f0bc9600640 1 Processor -- start 2024-05-27T12:35:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.171+0000 7f0bc9600640 1 -- start start 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.172+0000 7f0bc9600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bc4072630 0x7f0bc41541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.172+0000 7f0bc9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40b2f10 0x7f0bc414f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4163380 con 0x7f0bc415e920 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40ce100 con 0x7f0bc40b2f10 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc9600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc40cf030 con 0x7f0bc4072630 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48770/0 (socket says 172.21.15.94:48770) 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 -- 172.21.15.94:0/1341069569 learned_addr learned my addr 172.21.15.94:0/1341069569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bba000640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40b2f10 0x7f0bc414f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc2a00640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bc4072630 0x7f0bc41541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 -- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bc4072630 msgr2=0x7f0bc41541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bc4072630 0x7f0bc41541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 -- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40b2f10 msgr2=0x7f0bc414f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40b2f10 0x7f0bc414f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.173+0000 7f0bc3400640 1 -- 172.21.15.94:0/1341069569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41fcf60 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bc3400640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c780 tx=0x7f0bbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc004b60 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc004d00 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc41fc350 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00aa90 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.174+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41fc910 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.175+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4075460 con 0x7f0bc415e920 2024-05-27T12:35:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.176+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bbc00ac30 con 0x7f0bc415e920 2024-05-27T12:35:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.177+0000 7f0bbb400640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.177+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bbc08be20 con 0x7f0bc415e920 2024-05-27T12:35:20.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.178+0000 7f0bc2a00640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:20.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.179+0000 7f0bc2a00640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 0x7f0ba006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4034000 tx=0x7f0bb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:20.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.184+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bbc020070 con 0x7f0bc415e920 2024-05-27T12:35:20.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.421+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bc40730f0 con 0x7f0bc415e920 2024-05-27T12:35:20.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.422+0000 7f0bbb400640 1 -- 172.21.15.94:0/1341069569 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0bbc05a240 con 0x7f0bc415e920 2024-05-27T12:35:20.421 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T12:35:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 msgr2=0x7f0ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 0x7f0ba006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4034000 tx=0x7f0bb402e000 comp rx=0 tx=0).stop 2024-05-27T12:35:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 msgr2=0x7f0bc41614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:20.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c780 tx=0x7f0bbc00cc40 comp rx=0 tx=0).stop 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 shutdown_connections 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ba006c8f0 0x7f0ba006edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40b2f10 0x7f0bc414f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bc415e920 0x7f0bc41614f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 --2- 172.21.15.94:0/1341069569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bc4072630 0x7f0bc41541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.424+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 >> 172.21.15.94:0/1341069569 conn(0x7f0bc40b3830 msgr2=0x7f0bc40cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.425+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 shutdown_connections 2024-05-27T12:35:20.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20.425+0000 7f0bc9600640 1 -- 172.21.15.94:0/1341069569 wait complete. 2024-05-27T12:35:20.443 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T12:35:20.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T12:35:20.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T12:35:20.444 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-27T12:35:20.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:20.444 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-05-27T12:35:20.494 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:20.495 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether e6:74:08:05:41:39 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::f05f:10ff:fef1:11b0/64 scope link 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T12:35:20.496 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether e6:74:08:05:41:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T12:35:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::e474:8ff:fe05:4139/64 scope link 2024-05-27T12:35:20.497 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:20.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:20.497 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T12:35:20.573 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-05-27T12:35:20.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:20.578 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-05-27T12:35:20.633 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-05-27T12:35:20.633 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T12:35:20.633 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T12:35:20.633 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T12:35:20.633 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T12:35:20.634 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:20.634 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:20.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T12:35:20.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T12:35:20.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T12:35:20.635 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:20.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:20.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:20.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:20.851 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-05-27T12:35:20.897 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T12:35:20.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:20.898 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-05-27T12:35:20.944 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-27T12:35:20.945 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:20.945 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:20.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-27T12:35:20.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-27T12:35:20.945 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:21.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:21.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:21.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-27T12:35:21.245 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:21.245 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:21.245 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether d2:2a:d7:e8:20:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::d02a:d7ff:fee8:20f7/64 scope link 2024-05-27T12:35:21.246 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:21.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether d2:2a:d7:e8:20:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T12:35:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T12:35:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::d02a:d7ff:fee8:20f7/64 scope link 2024-05-27T12:35:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:21.339 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-27T12:35:21.339 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:21.339 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:21.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-27T12:35:21.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-27T12:35:21.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-27T12:35:21.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-27T12:35:21.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-27T12:35:21.340 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:21.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:21.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:21.610 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:21.610 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:35:21.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-05-27T12:35:21.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-05-27T12:35:21.611 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:35:21.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:21.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:21.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:21.823 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-27T12:35:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-27T12:35:21.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:21.869 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-27T12:35:21.964 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-27T12:35:21.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:21.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:22.092 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:22.092 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:22.127 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:22.127 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:22.127 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-05-27T12:35:22.127 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 2@b6a8d677-9e12-4245-aa4c-e7c6ae09d474.cephfs=/ 2024-05-27T12:35:22.127 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.94:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2024-05-27T12:35:22.129 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-27T12:35:22.129 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-05-27T12:35:22.129 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T12:35:22.129 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T12:35:22.130 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T12:35:22.134 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T12:35:22.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:22.134 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-27T12:35:22.212 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:22.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:22.332 INFO:teuthology.orchestra.run.smithi094.stdout:client4507 2024-05-27T12:35:22.332 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:22.332 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:22.387 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:22.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:22.503 INFO:teuthology.orchestra.run.smithi094.stdout:client4507 2024-05-27T12:35:22.504 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:22.617 INFO:teuthology.orchestra.run.smithi094.stdout:client4507 2024-05-27T12:35:22.617 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b6a8d677-9e12-4245-aa4c-e7c6ae09d474.client4507/status') 2024-05-27T12:35:22.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:22.742 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4507 (3)172.21.15.94:0/45600218 2024-05-27T12:35:22.742 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-05-27T12:35:22.742 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-05-27T12:35:22.742 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-05-27T12:35:22.742 INFO:teuthology.orchestra.run.smithi094.stderr:68 bytes copied, 8.001e-05 s, 850 kB/s 2024-05-27T12:35:22.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:22.755 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-27T12:35:22.755 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T12:35:22.755 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T12:35:22.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T12:35:22.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.948+0000 7f0c9be00640 1 Processor -- start 2024-05-27T12:35:22.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.948+0000 7f0c9be00640 1 -- start start 2024-05-27T12:35:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c9be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0c941553b0 0x7f0c940cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c9be00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9415b470 con 0x7f0c940d0260 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c9be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9415b710 con 0x7f0c94072ec0 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c9be00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0c9415b9f0 con 0x7f0c941553b0 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c98a00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0c940d0260 0x7f0c940727b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:37790/0 (socket says 172.21.15.94:37790) 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c98a00640 1 -- 172.21.15.94:0/702953417 learned_addr learned my addr 172.21.15.94:0/702953417 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c99400640 1 --2- 172.21.15.94:0/702953417 >> v2:172.21.15.94:3300/0 conn(0x7f0c941553b0 0x7f0c940cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.950+0000 7f0c93e00640 1 -- 172.21.15.94:0/702953417 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3438209840 0 0) 0x7f0c9415b470 con 0x7f0c940d0260 2024-05-27T12:35:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.951+0000 7f0c93e00640 1 -- 172.21.15.94:0/702953417 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c7c003520 con 0x7f0c940d0260 2024-05-27T12:35:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.951+0000 7f0c99400640 1 -- 172.21.15.94:0/702953417 >> v1:172.21.15.97:6789/0 conn(0x7f0c94072ec0 legacy=0x7f0c9415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.951+0000 7f0c99400640 1 -- 172.21.15.94:0/702953417 >> v1:172.21.15.94:6789/0 conn(0x7f0c940d0260 legacy=0x7f0c940727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.951+0000 7f0c99400640 1 -- 172.21.15.94:0/702953417 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9415bcd0 con 0x7f0c941553b0 2024-05-27T12:35:22.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.952+0000 7f0c99400640 1 --2- 172.21.15.94:0/702953417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c940cfd20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c880097a0 tx=0x7f0c8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c602ab639a7cd0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:22.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.952+0000 7f0c93e00640 1 -- 172.21.15.94:0/702953417 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88037070 con 0x7f0c941553b0 2024-05-27T12:35:22.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.952+0000 7f0c93e00640 1 -- 172.21.15.94:0/702953417 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c88032440 con 0x7f0c941553b0 2024-05-27T12:35:22.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.952+0000 7f0c93e00640 1 -- 172.21.15.94:0/702953417 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032810 con 0x7f0c941553b0 2024-05-27T12:35:22.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 -- 172.21.15.94:0/702953417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 msgr2=0x7f0c940cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:22.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 --2- 172.21.15.94:0/702953417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c940cfd20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c880097a0 tx=0x7f0c8802a1a0 comp rx=0 tx=0).stop 2024-05-27T12:35:22.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 -- 172.21.15.94:0/702953417 shutdown_connections 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 --2- 172.21.15.94:0/702953417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c940cfd20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 -- 172.21.15.94:0/702953417 >> 172.21.15.94:0/702953417 conn(0x7f0c940b3830 msgr2=0x7f0c940b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 -- 172.21.15.94:0/702953417 shutdown_connections 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.953+0000 7f0c9be00640 1 -- 172.21.15.94:0/702953417 wait complete. 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.955+0000 7f0c9be00640 1 Processor -- start 2024-05-27T12:35:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.955+0000 7f0c9be00640 1 -- start start 2024-05-27T12:35:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c94072ec0 0x7f0c94144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c940d0260 0x7f0c9415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c940c58f0 con 0x7f0c941553b0 2024-05-27T12:35:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94155790 con 0x7f0c94072ec0 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c9be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0c940cdb90 con 0x7f0c940d0260 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48786/0 (socket says 172.21.15.94:48786) 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 -- 172.21.15.94:0/2973790744 learned_addr learned my addr 172.21.15.94:0/2973790744 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99400640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c94072ec0 0x7f0c94144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:22.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c98a00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c940d0260 0x7f0c9415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 -- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c940d0260 msgr2=0x7f0c9415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c940d0260 0x7f0c9415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 -- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c94072ec0 msgr2=0x7f0c94144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.956+0000 7f0c99e00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c94072ec0 0x7f0c94144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c99e00640 1 -- 172.21.15.94:0/2973790744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c94157a20 con 0x7f0c941553b0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c99e00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c7d0 tx=0x7f0c8c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c003b70 con 0x7f0c941553b0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8c003d30 con 0x7f0c941553b0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00aa90 con 0x7f0c941553b0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c94157d00 con 0x7f0c941553b0 2024-05-27T12:35:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.957+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c941582c0 con 0x7f0c941553b0 2024-05-27T12:35:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.959+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c94075460 con 0x7f0c941553b0 2024-05-27T12:35:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.960+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c8c00ac30 con 0x7f0c941553b0 2024-05-27T12:35:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.960+0000 7f0c91600640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.961+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c8c08bb00 con 0x7f0c941553b0 2024-05-27T12:35:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.961+0000 7f0c99400640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.962+0000 7f0c99400640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 0x7f0c7406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c88034000 tx=0x7f0c8802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:22.965+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c8c059f20 con 0x7f0c941553b0 2024-05-27T12:35:23.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.211+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c940cd740 con 0x7f0c941553b0 2024-05-27T12:35:23.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.212+0000 7f0c91600640 1 -- 172.21.15.94:0/2973790744 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0c8c08be20 con 0x7f0c941553b0 2024-05-27T12:35:23.211 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 msgr2=0x7f0c7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 0x7f0c7406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c88034000 tx=0x7f0c8802e000 comp rx=0 tx=0).stop 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 msgr2=0x7f0c94146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c7d0 tx=0x7f0c8c00cc90 comp rx=0 tx=0).stop 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 shutdown_connections 2024-05-27T12:35:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0c7406c8f0 0x7f0c7406edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c941553b0 0x7f0c94146b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c940d0260 0x7f0c9415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 --2- 172.21.15.94:0/2973790744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c94072ec0 0x7f0c94144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.214+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 >> 172.21.15.94:0/2973790744 conn(0x7f0c940b3830 msgr2=0x7f0c940cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.215+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 shutdown_connections 2024-05-27T12:35:23.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:23.215+0000 7f0c9be00640 1 -- 172.21.15.94:0/2973790744 wait complete. 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T12:35:23.232 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T12:35:23.233 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T12:35:23.233 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T12:35:23.233 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-27T12:35:23.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:23.233 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-05-27T12:35:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-05-27T12:35:23.290 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-05-27T12:35:23.291 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:23.292 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T12:35:23.292 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:23.292 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:23.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-05-27T12:35:23.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-05-27T12:35:23.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-05-27T12:35:23.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T12:35:23.293 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:23.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:23.767 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T12:35:23.844 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-05-27T12:35:23.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:23.848 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-05-27T12:35:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-05-27T12:35:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T12:35:23.900 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T12:35:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T12:35:23.901 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T12:35:23.901 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:24.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:24.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:24.660 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-05-27T12:35:24.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:24.713 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-05-27T12:35:24.765 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:24.766 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:24.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T12:35:24.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-27T12:35:24.766 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:24.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:24.951 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-27T12:35:24.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-27T12:35:24.952 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:25.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:25.254 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:25.254 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:25.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-05-27T12:35:25.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T12:35:25.255 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:25.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:25.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:25.448 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-27T12:35:25.498 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-27T12:35:25.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:25.498 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-27T12:35:25.544 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-27T12:35:25.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:25.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:25.730 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:25.730 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-05-27T12:35:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@b6a8d677-9e12-4245-aa4c-e7c6ae09d474.cephfs=/ 2024-05-27T12:35:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.94:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2024-05-27T12:35:26.423 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-27T12:35:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-05-27T12:35:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T12:35:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T12:35:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T12:35:26.429 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T12:35:26.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:26.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-27T12:35:26.512 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:26.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:26.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:26.633 INFO:teuthology.orchestra.run.smithi097.stdout:client4418 2024-05-27T12:35:26.633 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:26.633 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:26.684 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:26.851 INFO:teuthology.orchestra.run.smithi097.stdout:client4418 2024-05-27T12:35:26.851 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:26.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:26.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:26.969 INFO:teuthology.orchestra.run.smithi097.stdout:client4418 2024-05-27T12:35:26.969 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b6a8d677-9e12-4245-aa4c-e7c6ae09d474.client4418/status') 2024-05-27T12:35:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:27.099 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4418 (3)192.168.0.1:0/1163486500 2024-05-27T12:35:27.099 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-05-27T12:35:27.100 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-05-27T12:35:27.100 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-05-27T12:35:27.100 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 6.1307e-05 s, 1.1 MB/s 2024-05-27T12:35:27.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:27.112 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-27T12:35:27.113 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T12:35:27.113 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-27T12:35:27.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T12:35:27.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.320+0000 7f2656c00640 1 Processor -- start 2024-05-27T12:35:27.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.320+0000 7f2656c00640 1 -- start start 2024-05-27T12:35:27.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26501460d0 0x7f2650160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2650071d60 0x7f26500721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26500ceae0 con 0x7f2650145cf0 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f26501602c0 con 0x7f2650071d60 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f26501605a0 con 0x7f26501460d0 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2655800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2650145cf0 0x7f26500ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:37792/0 (socket says 172.21.15.94:37792) 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2655800640 1 -- 172.21.15.94:0/1931150973 learned_addr learned my addr 172.21.15.94:0/1931150973 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:27.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.322+0000 7f2656200640 1 --2- 172.21.15.94:0/1931150973 >> v2:172.21.15.94:3300/0 conn(0x7f2650071d60 0x7f26500721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f2654e00640 1 --2- 172.21.15.94:0/1931150973 >> v2:172.21.15.97:3300/0 conn(0x7f26501460d0 0x7f2650160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f264fe00640 1 -- 172.21.15.94:0/1931150973 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205084827 0 0) 0x7f26500ceae0 con 0x7f2650145cf0 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f264fe00640 1 -- 172.21.15.94:0/1931150973 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2638003520 con 0x7f2650145cf0 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f2656200640 1 -- 172.21.15.94:0/1931150973 >> v2:172.21.15.97:3300/0 conn(0x7f26501460d0 msgr2=0x7f2650160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f2656200640 1 --2- 172.21.15.94:0/1931150973 >> v2:172.21.15.97:3300/0 conn(0x7f26501460d0 0x7f2650160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f2656200640 1 -- 172.21.15.94:0/1931150973 >> v1:172.21.15.94:6789/0 conn(0x7f2650145cf0 legacy=0x7f26500ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.323+0000 7f2656200640 1 -- 172.21.15.94:0/1931150973 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2650160880 con 0x7f2650071d60 2024-05-27T12:35:27.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.324+0000 7f2656200640 1 --2- 172.21.15.94:0/1931150973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2650071d60 0x7f26500721f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f265006c940 tx=0x7f264802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c582e82994a7b395 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:27.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.324+0000 7f264fe00640 1 -- 172.21.15.94:0/1931150973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648007740 con 0x7f2650071d60 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.324+0000 7f264fe00640 1 -- 172.21.15.94:0/1931150973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26480078e0 con 0x7f2650071d60 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.324+0000 7f264fe00640 1 -- 172.21.15.94:0/1931150973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648007cb0 con 0x7f2650071d60 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 -- 172.21.15.94:0/1931150973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2650071d60 msgr2=0x7f26500721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 --2- 172.21.15.94:0/1931150973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2650071d60 0x7f26500721f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f265006c940 tx=0x7f264802a680 comp rx=0 tx=0).stop 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 -- 172.21.15.94:0/1931150973 shutdown_connections 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 --2- 172.21.15.94:0/1931150973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2650071d60 0x7f26500721f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 --2- 172.21.15.94:0/1931150973 >> v2:172.21.15.97:3300/0 conn(0x7f26501460d0 0x7f2650160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 -- 172.21.15.94:0/1931150973 >> 172.21.15.94:0/1931150973 conn(0x7f26500b2160 msgr2=0x7f26500b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 -- 172.21.15.94:0/1931150973 shutdown_connections 2024-05-27T12:35:27.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.325+0000 7f2656c00640 1 -- 172.21.15.94:0/1931150973 wait complete. 2024-05-27T12:35:27.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.327+0000 7f2656c00640 1 Processor -- start 2024-05-27T12:35:27.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.327+0000 7f2656c00640 1 -- start start 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 0x7f265013bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2650145cf0 0x7f26500cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f265006eb00 con 0x7f26501460d0 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26500cedc0 con 0x7f2650071d60 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f265006ec80 con 0x7f2650145cf0 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48808/0 (socket says 172.21.15.94:48808) 2024-05-27T12:35:27.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 -- 172.21.15.94:0/3446418547 learned_addr learned my addr 172.21.15.94:0/3446418547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2655800640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 0x7f265013bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2654e00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2650145cf0 0x7f26500cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 -- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2650145cf0 msgr2=0x7f26500cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2650145cf0 0x7f26500cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 -- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 msgr2=0x7f265013bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 0x7f265013bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2656200640 1 -- 172.21.15.94:0/3446418547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500cb0c0 con 0x7f26501460d0 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.329+0000 7f2655800640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 0x7f265013bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:27.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f2656200640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f264802ac70 tx=0x7f2648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648040070 con 0x7f26501460d0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648031bc0 con 0x7f26501460d0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648009450 con 0x7f26501460d0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26501f7cd0 con 0x7f26501460d0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.330+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26501f8290 con 0x7f26501460d0 2024-05-27T12:35:27.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.331+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650075460 con 0x7f26501460d0 2024-05-27T12:35:27.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.332+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2648039040 con 0x7f26501460d0 2024-05-27T12:35:27.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.333+0000 7f264d600640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:27.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.334+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26480ad370 con 0x7f26501460d0 2024-05-27T12:35:27.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.334+0000 7f2655800640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:27.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.335+0000 7f2655800640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 0x7f263006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2644005880 tx=0x7f2644009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:27.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.338+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f264807b790 con 0x7f26501460d0 2024-05-27T12:35:27.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.579+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2650072e00 con 0x7f26501460d0 2024-05-27T12:35:27.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.580+0000 7f264d600640 1 -- 172.21.15.94:0/3446418547 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f264807b090 con 0x7f26501460d0 2024-05-27T12:35:27.579 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 msgr2=0x7f263006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 0x7f263006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2644005880 tx=0x7f2644009040 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 msgr2=0x7f26500c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f264802ac70 tx=0x7f2648004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 shutdown_connections 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f263006c8f0 0x7f263006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26501460d0 0x7f26500c83f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2650145cf0 0x7f26500cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 --2- 172.21.15.94:0/3446418547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2650071d60 0x7f265013bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 >> 172.21.15.94:0/3446418547 conn(0x7f26500b2160 msgr2=0x7f2650072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:27.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 shutdown_connections 2024-05-27T12:35:27.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:27.583+0000 7f2656c00640 1 -- 172.21.15.94:0/3446418547 wait complete. 2024-05-27T12:35:27.600 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T12:35:27.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T12:35:27.601 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-27T12:35:27.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:27.601 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:27.660 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 4e:19:96:0f:fd:e2 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:35:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c49d:7cff:fe05:942a/64 scope link 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 4e:19:96:0f:fd:e2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::4c19:96ff:fe0f:fde2/64 scope link 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:27.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:27.662 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T12:35:27.742 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-05-27T12:35:27.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:27.747 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-05-27T12:35:27.800 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-05-27T12:35:27.800 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T12:35:27.801 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T12:35:27.801 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T12:35:27.801 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T12:35:27.802 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:27.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:28.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:28.007 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-05-27T12:35:28.052 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-27T12:35:28.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:28.053 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-05-27T12:35:28.151 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-27T12:35:28.152 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:28.152 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:28.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-27T12:35:28.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-27T12:35:28.152 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:28.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:28.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:28.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:28.407 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 16:0a:72:ec:92:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::140a:72ff:feec:9247/64 scope link 2024-05-27T12:35:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:28.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:35:28.488 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 16:0a:72:ec:92:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T12:35:28.488 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T12:35:28.488 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.488 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::140a:72ff:feec:9247/64 scope link 2024-05-27T12:35:28.488 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:35:28.491 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-27T12:35:28.491 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-27T12:35:28.492 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:28.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:35:28.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:35:28.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:28.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:28.758 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:35:28.758 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-05-27T12:35:28.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2024-05-27T12:35:28.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2024-05-27T12:35:28.758 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-05-27T12:35:28.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:29.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:29.015 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-27T12:35:29.070 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-27T12:35:29.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:35:29.070 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-27T12:35:29.121 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-27T12:35:29.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:29.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:29.245 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:29.245 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:29.278 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:35:29.278 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:35:29.278 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-05-27T12:35:29.278 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 3@b6a8d677-9e12-4245-aa4c-e7c6ae09d474.cephfs=/ 2024-05-27T12:35:29.278 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.94:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2024-05-27T12:35:29.281 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-27T12:35:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-05-27T12:35:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T12:35:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T12:35:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T12:35:29.286 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T12:35:29.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:29.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-27T12:35:29.359 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:29.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:29.475 INFO:teuthology.orchestra.run.smithi097.stdout:client4519 2024-05-27T12:35:29.475 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:29.475 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:29.526 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:29.646 INFO:teuthology.orchestra.run.smithi097.stdout:client4519 2024-05-27T12:35:29.646 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:35:29.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:35:29.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:29.762 INFO:teuthology.orchestra.run.smithi097.stdout:client4519 2024-05-27T12:35:29.762 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b6a8d677-9e12-4245-aa4c-e7c6ae09d474.client4519/status') 2024-05-27T12:35:29.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:35:29.894 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4519 (3)192.168.0.2:0/3654399695 2024-05-27T12:35:29.894 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-05-27T12:35:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-05-27T12:35:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-05-27T12:35:29.894 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 9.5393e-05 s, 723 kB/s 2024-05-27T12:35:29.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-05-27T12:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:35:29.907 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-27T12:35:29.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.123+0000 7ff240800640 1 Processor -- start 2024-05-27T12:35:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.123+0000 7ff240800640 1 -- start start 2024-05-27T12:35:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff240800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 0x7ff2380c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff240800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23806f450 con 0x7ff23815a250 2024-05-27T12:35:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff240800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff238160360 con 0x7ff2380cf400 2024-05-27T12:35:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff240800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff238160f70 con 0x7ff2380cf840 2024-05-27T12:35:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff23de00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff23815a250 0x7ff23806ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54462/0 (socket says 172.21.15.94:54462) 2024-05-27T12:35:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff23de00640 1 -- 172.21.15.94:0/3282490566 learned_addr learned my addr 172.21.15.94:0/3282490566 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:30.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 580148358 0 0) 0x7ff23806f450 con 0x7ff23815a250 2024-05-27T12:35:30.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff220003520 con 0x7ff23815a250 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.125+0000 7ff23e800640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 0x7ff2380c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3386887233 0 0) 0x7ff220003520 con 0x7ff23815a250 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 msgr2=0x7ff2380c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 0x7ff2380c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 >> v1:172.21.15.97:6789/0 conn(0x7ff2380cf400 legacy=0x7ff2380c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff238160640 con 0x7ff23815a250 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7ff22c001690 con 0x7ff23815a250 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 >> v1:172.21.15.94:6789/0 conn(0x7ff23815a250 legacy=0x7ff23806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220003ac0 0x7ff220003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff220004800 0x7ff220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 0x7ff22000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff220005510 con 0x7ff220004800 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff22000bf00 con 0x7ff220003ac0 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff22000c080 con 0x7ff2380cf840 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.126+0000 7ff23d400640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff220004800 0x7ff220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23de00640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220003ac0 0x7ff220003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23d400640 1 -- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 msgr2=0x7ff22000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23d400640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2380cf840 0x7ff22000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23d400640 1 -- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220003ac0 msgr2=0x7ff220003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23d400640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220003ac0 0x7ff220003ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.127+0000 7ff23d400640 1 -- 172.21.15.94:0/3282490566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff220003520 con 0x7ff220004800 2024-05-27T12:35:30.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.128+0000 7ff23d400640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff220004800 0x7ff220004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff228001750 tx=0x7ff220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dff7cbac8b01c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.128+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228001370 con 0x7ff220004800 2024-05-27T12:35:30.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.128+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2280052d0 con 0x7ff220004800 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.128+0000 7ff23ca00640 1 -- 172.21.15.94:0/3282490566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22802abe0 con 0x7ff220004800 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 -- 172.21.15.94:0/3282490566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff220004800 msgr2=0x7ff220004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 --2- 172.21.15.94:0/3282490566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff220004800 0x7ff220004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff228001750 tx=0x7ff220005830 comp rx=0 tx=0).stop 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff23e800640 1 -- 172.21.15.94:0/3282490566 reap_dead start 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 -- 172.21.15.94:0/3282490566 shutdown_connections 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 -- 172.21.15.94:0/3282490566 >> 172.21.15.94:0/3282490566 conn(0x7ff2380b3830 msgr2=0x7ff2380b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 -- 172.21.15.94:0/3282490566 shutdown_connections 2024-05-27T12:35:30.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.129+0000 7ff240800640 1 -- 172.21.15.94:0/3282490566 wait complete. 2024-05-27T12:35:30.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.131+0000 7ff240800640 1 Processor -- start 2024-05-27T12:35:30.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.131+0000 7ff240800640 1 -- start start 2024-05-27T12:35:30.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220004800 0x7ff23815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff238157da0 0x7ff238158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2380cfc90 con 0x7ff23815cd20 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff238160c00 con 0x7ff220004800 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff240800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff23815a630 con 0x7ff238157da0 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56646/0 (socket says 172.21.15.94:56646) 2024-05-27T12:35:30.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 -- 172.21.15.94:0/332971479 learned_addr learned my addr 172.21.15.94:0/332971479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23de00640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220004800 0x7ff23815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 -- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff238157da0 msgr2=0x7ff238158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff238157da0 0x7ff238158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 -- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220004800 msgr2=0x7ff23815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.132+0000 7ff23d400640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220004800 0x7ff23815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff23d400640 1 -- 172.21.15.94:0/332971479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23815d2e0 con 0x7ff23815cd20 2024-05-27T12:35:30.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff23d400640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff228008fd0 tx=0x7ff228001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228037040 con 0x7ff23815cd20 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff228005480 con 0x7ff23815cd20 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228007a30 con 0x7ff23815cd20 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23815d5c0 con 0x7ff23815cd20 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.133+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23815db80 con 0x7ff23815cd20 2024-05-27T12:35:30.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.135+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238075460 con 0x7ff23815cd20 2024-05-27T12:35:30.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.136+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff228044020 con 0x7ff23815cd20 2024-05-27T12:35:30.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.137+0000 7ff232000640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.137+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2280ab8b0 con 0x7ff23815cd20 2024-05-27T12:35:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.138+0000 7ff23de00640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.139+0000 7ff23de00640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0011e0 tx=0x7ff22c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.142+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff228079c40 con 0x7ff23815cd20 2024-05-27T12:35:30.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.343+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff23806e1c0 con 0x7ff23815cd20 2024-05-27T12:35:30.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.344+0000 7ff232000640 1 -- 172.21.15.94:0/332971479 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7ff228079540 con 0x7ff23815cd20 2024-05-27T12:35:30.344 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:30.344 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.346+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 msgr2=0x7ff21c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.346+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0011e0 tx=0x7ff22c009040 comp rx=0 tx=0).stop 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.346+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 msgr2=0x7ff238143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.346+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff228008fd0 tx=0x7ff228001120 comp rx=0 tx=0).stop 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 shutdown_connections 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff238157da0 0x7ff238158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23815cd20 0x7ff238143ee0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 --2- 172.21.15.94:0/332971479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff220004800 0x7ff23815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 >> 172.21.15.94:0/332971479 conn(0x7ff2380b3830 msgr2=0x7ff23815abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 shutdown_connections 2024-05-27T12:35:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.347+0000 7ff240800640 1 -- 172.21.15.94:0/332971479 wait complete. 2024-05-27T12:35:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:30.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:30.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.541+0000 7ffa12000640 1 Processor -- start 2024-05-27T12:35:30.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.541+0000 7ffa12000640 1 -- start start 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.542+0000 7ffa12000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa0c0b4710 0x7ffa0c1624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.542+0000 7ffa12000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa0c0adb10 0x7ffa0c162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.542+0000 7ffa12000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa0c15ed10 con 0x7ffa0c15e930 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.542+0000 7ffa12000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c072930 con 0x7ffa0c0adb10 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.542+0000 7ffa12000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c072000 con 0x7ffa0c0b4710 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffa0c0adb10 0x7ffa0c162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0b400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ffa0c15e930 0x7ffa0c0b4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54470/0 (socket says 172.21.15.94:54470) 2024-05-27T12:35:30.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0b400640 1 -- 172.21.15.94:0/2622924453 learned_addr learned my addr 172.21.15.94:0/2622924453 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:30.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0aa00640 1 --2- 172.21.15.94:0/2622924453 >> v2:172.21.15.97:3300/0 conn(0x7ffa0c0b4710 0x7ffa0c1624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0be00640 1 -- 172.21.15.94:0/2622924453 >> v2:172.21.15.97:3300/0 conn(0x7ffa0c0b4710 msgr2=0x7ffa0c1624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0be00640 1 --2- 172.21.15.94:0/2622924453 >> v2:172.21.15.97:3300/0 conn(0x7ffa0c0b4710 0x7ffa0c1624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0be00640 1 -- 172.21.15.94:0/2622924453 >> v1:172.21.15.94:6789/0 conn(0x7ffa0c15e930 legacy=0x7ffa0c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.543+0000 7ffa0be00640 1 -- 172.21.15.94:0/2622924453 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0722e0 con 0x7ffa0c0adb10 2024-05-27T12:35:30.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.544+0000 7ffa0be00640 1 --2- 172.21.15.94:0/2622924453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c162a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0af960 tx=0x7ffa0402a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dffa5c5bdbb0021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.544+0000 7ffa0a000640 1 -- 172.21.15.94:0/2622924453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ffa0c0adb10 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.544+0000 7ffa0a000640 1 -- 172.21.15.94:0/2622924453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0402aca0 con 0x7ffa0c0adb10 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa0a000640 1 -- 172.21.15.94:0/2622924453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04032660 con 0x7ffa0c0adb10 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 -- 172.21.15.94:0/2622924453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0adb10 msgr2=0x7ffa0c162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 --2- 172.21.15.94:0/2622924453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c162a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0af960 tx=0x7ffa0402a010 comp rx=0 tx=0).stop 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 -- 172.21.15.94:0/2622924453 shutdown_connections 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 --2- 172.21.15.94:0/2622924453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c162a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 --2- 172.21.15.94:0/2622924453 >> v2:172.21.15.97:3300/0 conn(0x7ffa0c0b4710 0x7ffa0c1624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 -- 172.21.15.94:0/2622924453 >> 172.21.15.94:0/2622924453 conn(0x7ffa0c0b2240 msgr2=0x7ffa0c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 -- 172.21.15.94:0/2622924453 shutdown_connections 2024-05-27T12:35:30.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.545+0000 7ffa12000640 1 -- 172.21.15.94:0/2622924453 wait complete. 2024-05-27T12:35:30.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.547+0000 7ffa12000640 1 Processor -- start 2024-05-27T12:35:30.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.547+0000 7ffa12000640 1 -- start start 2024-05-27T12:35:30.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa0c15e930 0x7ffa0c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0b3850 con 0x7ffa0c0b4710 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0b36d0 con 0x7ffa0c0adb10 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa12000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c06fd20 con 0x7ffa0c15e930 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa0aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa0aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56666/0 (socket says 172.21.15.94:56666) 2024-05-27T12:35:30.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.548+0000 7ffa0aa00640 1 -- 172.21.15.94:0/242792434 learned_addr learned my addr 172.21.15.94:0/242792434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:30.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0be00640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa0c15e930 0x7ffa0c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0b400640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 -- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa0c15e930 msgr2=0x7ffa0c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa0c15e930 0x7ffa0c06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 -- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0c0adb10 msgr2=0x7ffa0c146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c146db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 -- 172.21.15.94:0/242792434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0cde30 con 0x7ffa0c0b4710 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa0aa00640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc007ef0 tx=0x7ff9fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc014620 con 0x7ffa0c0b4710 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc019070 con 0x7ffa0c0b4710 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc014a00 con 0x7ffa0c0b4710 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.549+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c0cdb50 con 0x7ffa0c0b4710 2024-05-27T12:35:30.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.550+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c151900 con 0x7ffa0c0b4710 2024-05-27T12:35:30.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.551+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c068a70 con 0x7ffa0c0b4710 2024-05-27T12:35:30.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.552+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9fc014ba0 con 0x7ffa0c0b4710 2024-05-27T12:35:30.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.554+0000 7ffa03400640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.554+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9fc08c480 con 0x7ffa0c0b4710 2024-05-27T12:35:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.554+0000 7ffa0b400640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.555+0000 7ffa0b400640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 0x7ff9e806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.558+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9fc05a890 con 0x7ffa0c0b4710 2024-05-27T12:35:30.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.759+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa0c06e5d0 con 0x7ffa0c0b4710 2024-05-27T12:35:30.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.760+0000 7ffa03400640 1 -- 172.21.15.94:0/242792434 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff9fc05a190 con 0x7ffa0c0b4710 2024-05-27T12:35:30.760 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 msgr2=0x7ff9e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 0x7ff9e806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4009040 comp rx=0 tx=0).stop 2024-05-27T12:35:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 msgr2=0x7ffa0c06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc007ef0 tx=0x7ff9fc00e4b0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 shutdown_connections 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa0c15e930 0x7ffa0c06be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0b4710 0x7ffa0c06b930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 --2- 172.21.15.94:0/242792434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0c0adb10 0x7ffa0c146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 >> 172.21.15.94:0/242792434 conn(0x7ffa0c0b2240 msgr2=0x7ffa0c150940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 shutdown_connections 2024-05-27T12:35:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.763+0000 7ffa12000640 1 -- 172.21.15.94:0/242792434 wait complete. 2024-05-27T12:35:30.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.958+0000 7fed72400640 1 Processor -- start 2024-05-27T12:35:30.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.958+0000 7fed72400640 1 -- start start 2024-05-27T12:35:30.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.959+0000 7fed72400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fed6c0ba3f0 0x7fed6c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed72400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c0d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed72400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed6c0b7840 con 0x7fed6c15e900 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed72400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fed6c0b6b40 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed72400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fed6c067f30 con 0x7fed6c0b66d0 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fed6c0ba3f0 0x7fed6c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fed6c0ba3f0 0x7fed6c0b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56682/0 (socket says 172.21.15.94:56682) 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 -- 172.21.15.94:0/3140431150 learned_addr learned my addr 172.21.15.94:0/3140431150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 -- 172.21.15.94:0/3140431150 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 msgr2=0x7fed6c0d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed71a00640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c0d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c0d3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 -- 172.21.15.94:0/3140431150 >> v1:172.21.15.97:6789/0 conn(0x7fed6c15e900 legacy=0x7fed6c067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:30.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.960+0000 7fed63e00640 1 -- 172.21.15.94:0/3140431150 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c068210 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.961+0000 7fed63e00640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c0b6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71cf4ab9f963b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed6be00640 1 -- 172.21.15.94:0/3140431150 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54037070 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed6be00640 1 -- 172.21.15.94:0/3140431150 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed54032440 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed6be00640 1 -- 172.21.15.94:0/3140431150 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54032810 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 -- 172.21.15.94:0/3140431150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c0ba3f0 msgr2=0x7fed6c0b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c0b6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402a2a0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 -- 172.21.15.94:0/3140431150 shutdown_connections 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c0d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 --2- 172.21.15.94:0/3140431150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c0b6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 -- 172.21.15.94:0/3140431150 >> 172.21.15.94:0/3140431150 conn(0x7fed6c0d1190 msgr2=0x7fed6c0d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.962+0000 7fed72400640 1 -- 172.21.15.94:0/3140431150 shutdown_connections 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.963+0000 7fed72400640 1 -- 172.21.15.94:0/3140431150 wait complete. 2024-05-27T12:35:30.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.964+0000 7fed72400640 1 Processor -- start 2024-05-27T12:35:30.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.964+0000 7fed72400640 1 -- start start 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c0d2df0 con 0x7fed6c15e900 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c0d2b10 con 0x7fed6c0ba3f0 2024-05-27T12:35:30.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed72400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fed6c15ece0 con 0x7fed6c0b66d0 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed71a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed71a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56688/0 (socket says 172.21.15.94:56688) 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.965+0000 7fed71a00640 1 -- 172.21.15.94:0/748600375 learned_addr learned my addr 172.21.15.94:0/748600375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed63e00640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71000640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 -- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 msgr2=0x7fed6c161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 -- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6c0ba3f0 msgr2=0x7fed6c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 -- 172.21.15.94:0/748600375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c160cc0 con 0x7fed6c15e900 2024-05-27T12:35:30.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71000640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed71a00640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cd70 tx=0x7fed6400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64003b90 con 0x7fed6c15e900 2024-05-27T12:35:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed64003d30 con 0x7fed6c15e900 2024-05-27T12:35:30.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1f8dd0 con 0x7fed6c15e900 2024-05-27T12:35:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.966+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64004240 con 0x7fed6c15e900 2024-05-27T12:35:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.967+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c160f80 con 0x7fed6c15e900 2024-05-27T12:35:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.968+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6c075460 con 0x7fed6c15e900 2024-05-27T12:35:30.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.969+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed640043e0 con 0x7fed6c15e900 2024-05-27T12:35:30.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.969+0000 7fed69600640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 0x7fed4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:30.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.969+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed6408c6d0 con 0x7fed6c15e900 2024-05-27T12:35:30.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.970+0000 7fed71000640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 0x7fed4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:30.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.971+0000 7fed71000640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 0x7fed4806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00a830 tx=0x7fed5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:30.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:30.973+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed6405aa60 con 0x7fed6c15e900 2024-05-27T12:35:31.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.164+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fed6c158a10 con 0x7fed6c15e900 2024-05-27T12:35:31.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.165+0000 7fed69600640 1 -- 172.21.15.94:0/748600375 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fed6405a360 con 0x7fed6c15e900 2024-05-27T12:35:31.165 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:31.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.167+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 msgr2=0x7fed4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.167+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 0x7fed4806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00a830 tx=0x7fed5c008040 comp rx=0 tx=0).stop 2024-05-27T12:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.167+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 msgr2=0x7fed6c1f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.167+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cd70 tx=0x7fed6400a660 comp rx=0 tx=0).stop 2024-05-27T12:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 shutdown_connections 2024-05-27T12:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fed4806c8f0 0x7fed4806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6c0ba3f0 0x7fed6c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed6c15e900 0x7fed6c1f8310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 --2- 172.21.15.94:0/748600375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fed6c0b66d0 0x7fed6c161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 >> 172.21.15.94:0/748600375 conn(0x7fed6c0d1190 msgr2=0x7fed6c0b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 shutdown_connections 2024-05-27T12:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.168+0000 7fed72400640 1 -- 172.21.15.94:0/748600375 wait complete. 2024-05-27T12:35:31.184 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:31.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.375+0000 7fba9e600640 1 Processor -- start 2024-05-27T12:35:31.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.375+0000 7fba9e600640 1 -- start start 2024-05-27T12:35:31.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 0x7fba980b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9e600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba980b66d0 0x7fba980d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9e600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980b7840 con 0x7fba9815e900 2024-05-27T12:35:31.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9e600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fba98067f30 con 0x7fba980b66d0 2024-05-27T12:35:31.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fba98068210 con 0x7fba980ba3f0 2024-05-27T12:35:31.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9d200640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fba9815e900 0x7fba98067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54484/0 (socket says 172.21.15.94:54484) 2024-05-27T12:35:31.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.376+0000 7fba9d200640 1 -- 172.21.15.94:0/3252697602 learned_addr learned my addr 172.21.15.94:0/3252697602 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:31.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478851567 0 0) 0x7fba8c001110 con 0x7fba9815e900 2024-05-27T12:35:31.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba80003520 con 0x7fba9815e900 2024-05-27T12:35:31.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97e00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 0x7fba980b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba9dc00640 1 --2- 172.21.15.94:0/3252697602 >> v2:172.21.15.97:3300/0 conn(0x7fba980b66d0 0x7fba980d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2350111987 0 0) 0x7fba80003520 con 0x7fba9815e900 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 msgr2=0x7fba980b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 0x7fba980b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 >> v2:172.21.15.97:3300/0 conn(0x7fba980b66d0 msgr2=0x7fba980d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 --2- 172.21.15.94:0/3252697602 >> v2:172.21.15.97:3300/0 conn(0x7fba980b66d0 0x7fba980d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980684f0 con 0x7fba9815e900 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fba8c001840 con 0x7fba9815e900 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 >> v1:172.21.15.94:6789/0 conn(0x7fba9815e900 legacy=0x7fba98067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 0x7fba800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba80004d40 0x7fba800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba98067f30 con 0x7fba80003fe0 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba98068210 con 0x7fba80004d40 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fba8000bb50 con 0x7fba980ba3f0 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.377+0000 7fba97e00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba9dc00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba80004d40 0x7fba800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba97e00640 1 -- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 msgr2=0x7fba800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba97e00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba980ba3f0 0x7fba800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba97e00640 1 -- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba80004d40 msgr2=0x7fba800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba97e00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba80004d40 0x7fba800051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.378+0000 7fba97e00640 1 -- 172.21.15.94:0/3252697602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba80003520 con 0x7fba80003fe0 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.379+0000 7fba97e00640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba80004800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ba30 tx=0x7fba80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3269647180ddd451 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.379+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84030070 con 0x7fba80003fe0 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.379+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8402ae10 con 0x7fba80003fe0 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.379+0000 7fba97400640 1 -- 172.21.15.94:0/3252697602 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400b600 con 0x7fba80003fe0 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 -- 172.21.15.94:0/3252697602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 msgr2=0x7fba80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 --2- 172.21.15.94:0/3252697602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba80004800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ba30 tx=0x7fba80005830 comp rx=0 tx=0).stop 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9dc00640 1 -- 172.21.15.94:0/3252697602 reap_dead start 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 -- 172.21.15.94:0/3252697602 shutdown_connections 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 -- 172.21.15.94:0/3252697602 >> 172.21.15.94:0/3252697602 conn(0x7fba980d1190 msgr2=0x7fba980d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 -- 172.21.15.94:0/3252697602 shutdown_connections 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.380+0000 7fba9e600640 1 -- 172.21.15.94:0/3252697602 wait complete. 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.382+0000 7fba9e600640 1 Processor -- start 2024-05-27T12:35:31.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.382+0000 7fba9e600640 1 -- start start 2024-05-27T12:35:31.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba981e7bd0 0x7fba980ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba981e8030 0x7fba980cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba9815ece0 con 0x7fba80003fe0 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba980d2df0 con 0x7fba981e8030 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9e600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fba980b6190 con 0x7fba981e7bd0 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56706/0 (socket says 172.21.15.94:56706) 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 -- 172.21.15.94:0/2685564476 learned_addr learned my addr 172.21.15.94:0/2685564476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 -- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba981e7bd0 msgr2=0x7fba980ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba981e7bd0 0x7fba980ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 -- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba981e8030 msgr2=0x7fba980cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba981e8030 0x7fba980cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 -- 172.21.15.94:0/2685564476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba981600a0 con 0x7fba80003fe0 2024-05-27T12:35:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.383+0000 7fba9d200640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c7a0 tx=0x7fba8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.384+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c018070 con 0x7fba80003fe0 2024-05-27T12:35:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.384+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c0074e0 con 0x7fba80003fe0 2024-05-27T12:35:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.384+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba98160380 con 0x7fba80003fe0 2024-05-27T12:35:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.384+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c004a20 con 0x7fba80003fe0 2024-05-27T12:35:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.384+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba98160940 con 0x7fba80003fe0 2024-05-27T12:35:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.385+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba980b8ac0 con 0x7fba80003fe0 2024-05-27T12:35:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.386+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba8c01b070 con 0x7fba80003fe0 2024-05-27T12:35:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.387+0000 7fba94c00640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.387+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba8c08b0d0 con 0x7fba80003fe0 2024-05-27T12:35:31.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.388+0000 7fba97e00640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.389+0000 7fba97e00640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 0x7fba7806ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ba00 tx=0x7fba8402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.392+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba980b8ac0 con 0x7fba80003fe0 2024-05-27T12:35:31.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.579+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba98075460 con 0x7fba80003fe0 2024-05-27T12:35:31.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.580+0000 7fba94c00640 1 -- 172.21.15.94:0/2685564476 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fba980b7840 con 0x7fba80003fe0 2024-05-27T12:35:31.580 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 msgr2=0x7fba7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 0x7fba7806ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba8400ba00 tx=0x7fba8402c000 comp rx=0 tx=0).stop 2024-05-27T12:35:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 msgr2=0x7fba980ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c7a0 tx=0x7fba8c00cc60 comp rx=0 tx=0).stop 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 shutdown_connections 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fba7806c920 0x7fba7806ede0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba981e8030 0x7fba980cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba981e7bd0 0x7fba980ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 --2- 172.21.15.94:0/2685564476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba80003fe0 0x7fba980ca350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 >> 172.21.15.94:0/2685564476 conn(0x7fba980d1190 msgr2=0x7fba9815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.582+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 shutdown_connections 2024-05-27T12:35:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.583+0000 7fba9e600640 1 -- 172.21.15.94:0/2685564476 wait complete. 2024-05-27T12:35:31.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.781+0000 7f6017200640 1 Processor -- start 2024-05-27T12:35:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.781+0000 7f6017200640 1 -- start start 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.782+0000 7f6017200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f60180cdb90 0x7f6018071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.782+0000 7f6017200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601813e9e0 con 0x7f601813e600 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.782+0000 7f6017200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601815ba50 con 0x7f601806f5e0 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.782+0000 7f6017200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f601815bd10 con 0x7f60180cdb90 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6015e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f601813e600 0x7f601806eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54486/0 (socket says 172.21.15.94:54486) 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6015e00640 1 -- 172.21.15.94:0/3321061329 learned_addr learned my addr 172.21.15.94:0/3321061329 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6016800640 1 --2- 172.21.15.94:0/3321061329 >> v2:172.21.15.94:3300/0 conn(0x7f60180cdb90 0x7f6018071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6015400640 1 -- 172.21.15.94:0/3321061329 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916619039 0 0) 0x7f6008001110 con 0x7f601813e600 2024-05-27T12:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6016800640 1 -- 172.21.15.94:0/3321061329 >> v1:172.21.15.97:6789/0 conn(0x7f601806f5e0 legacy=0x7f60180cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6016800640 1 -- 172.21.15.94:0/3321061329 >> v1:172.21.15.94:6789/0 conn(0x7f601813e600 legacy=0x7f601806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.783+0000 7f6016800640 1 -- 172.21.15.94:0/3321061329 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601815bff0 con 0x7f60180cdb90 2024-05-27T12:35:31.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.784+0000 7f6016800640 1 --2- 172.21.15.94:0/3321061329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f6018071d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d90 tx=0x7f601002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14607624c0576c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.784+0000 7f6015400640 1 -- 172.21.15.94:0/3321061329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010007740 con 0x7f60180cdb90 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.784+0000 7f6015400640 1 -- 172.21.15.94:0/3321061329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60100078e0 con 0x7f60180cdb90 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.784+0000 7f6015400640 1 -- 172.21.15.94:0/3321061329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010007cb0 con 0x7f60180cdb90 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 -- 172.21.15.94:0/3321061329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 msgr2=0x7f6018071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 --2- 172.21.15.94:0/3321061329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f6018071d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d90 tx=0x7f601002a6a0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 -- 172.21.15.94:0/3321061329 shutdown_connections 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 --2- 172.21.15.94:0/3321061329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f6018071d60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 -- 172.21.15.94:0/3321061329 >> 172.21.15.94:0/3321061329 conn(0x7f60180b2160 msgr2=0x7f60180b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 -- 172.21.15.94:0/3321061329 shutdown_connections 2024-05-27T12:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.785+0000 7f6017200640 1 -- 172.21.15.94:0/3321061329 wait complete. 2024-05-27T12:35:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.787+0000 7f6017200640 1 Processor -- start 2024-05-27T12:35:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.787+0000 7f6017200640 1 -- start start 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 0x7f60180b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f601813e600 0x7f601815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f601815c920 con 0x7f60180cdb90 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f601815c5b0 con 0x7f601806f5e0 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6017200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f6018141030 con 0x7f601813e600 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f600fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f600fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56724/0 (socket says 172.21.15.94:56724) 2024-05-27T12:35:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f600fe00640 1 -- 172.21.15.94:0/2136484912 learned_addr learned my addr 172.21.15.94:0/2136484912 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.788+0000 7f6015e00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 0x7f60180b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 -- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f601813e600 msgr2=0x7f601815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f6016800640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f601813e600 0x7f601815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f601813e600 0x7f601815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 -- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 msgr2=0x7f60180b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 0x7f60180b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 -- 172.21.15.94:0/2136484912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180b33f0 con 0x7f60180cdb90 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f6015e00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 0x7f60180b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600fe00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f600000c3f0 tx=0x7f600000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000007020 con 0x7f60180cdb90 2024-05-27T12:35:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60000071c0 con 0x7f60180cdb90 2024-05-27T12:35:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60180b36d0 con 0x7f60180cdb90 2024-05-27T12:35:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000007590 con 0x7f60180cdb90 2024-05-27T12:35:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.789+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60180b3c90 con 0x7f60180cdb90 2024-05-27T12:35:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.790+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60180b3e90 con 0x7f60180cdb90 2024-05-27T12:35:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.791+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6000019020 con 0x7f60180cdb90 2024-05-27T12:35:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.792+0000 7f600e000640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 0x7f5ff406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.792+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f600008b3e0 con 0x7f60180cdb90 2024-05-27T12:35:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.793+0000 7f6015e00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 0x7f5ff406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:31.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.794+0000 7f6015e00640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 0x7f5ff406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:31.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.796+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60180b3e90 con 0x7f60180cdb90 2024-05-27T12:35:31.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:31.997+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60180d00f0 con 0x7f60180cdb90 2024-05-27T12:35:31.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.000+0000 7f600e000640 1 -- 172.21.15.94:0/2136484912 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f6000059770 con 0x7f60180cdb90 2024-05-27T12:35:32.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:32.000 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:32.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.002+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 msgr2=0x7f5ff406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 0x7f5ff406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).stop 2024-05-27T12:35:32.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 msgr2=0x7f60180aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f600000c3f0 tx=0x7f600000c8b0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 shutdown_connections 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5ff406c8f0 0x7f5ff406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f601813e600 0x7f601815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60180cdb90 0x7f60180aeeb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 --2- 172.21.15.94:0/2136484912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601806f5e0 0x7f60180b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 >> 172.21.15.94:0/2136484912 conn(0x7f60180b2160 msgr2=0x7f601813ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 shutdown_connections 2024-05-27T12:35:32.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.003+0000 7f6017200640 1 -- 172.21.15.94:0/2136484912 wait complete. 2024-05-27T12:35:32.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:32.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:32.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.224+0000 7fc0e4a00640 1 Processor -- start 2024-05-27T12:35:32.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.224+0000 7fc0e4a00640 1 -- start start 2024-05-27T12:35:32.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0e4a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 0x7fc0e0071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0e4a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0e015d2f0 0x7fc0e0140a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0e4a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e006dc50 con 0x7fc0e0072260 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0e4a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e00726c0 con 0x7fc0e015d2f0 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0e4a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e013fcd0 con 0x7fc0e013f480 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0dde00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc0e0072260 0x7fc0e015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54492/0 (socket says 172.21.15.94:54492) 2024-05-27T12:35:32.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0dde00640 1 -- 172.21.15.94:0/4273744466 learned_addr learned my addr 172.21.15.94:0/4273744466 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:32.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.226+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049824583 0 0) 0x7fc0c8001110 con 0x7fc0e0072260 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0c4003520 con 0x7fc0e0072260 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0df200640 1 --2- 172.21.15.94:0/4273744466 >> v2:172.21.15.97:3300/0 conn(0x7fc0e015d2f0 0x7fc0e0140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0de800640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 0x7fc0e0071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1236165270 0 0) 0x7fc0c4003520 con 0x7fc0e0072260 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 msgr2=0x7fc0e0071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 0x7fc0e0071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 >> v2:172.21.15.97:3300/0 conn(0x7fc0e015d2f0 msgr2=0x7fc0e0140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 --2- 172.21.15.94:0/4273744466 >> v2:172.21.15.97:3300/0 conn(0x7fc0e015d2f0 0x7fc0e0140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e015d950 con 0x7fc0e0072260 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fc0c8001840 con 0x7fc0e0072260 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 >> v1:172.21.15.94:6789/0 conn(0x7fc0e0072260 legacy=0x7fc0e015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40039b0 0x7fc0c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c4004800 0x7fc0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 0x7fc0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e013fcd0 con 0x7fc0c40039b0 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c400b9f0 con 0x7fc0c4004800 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c400bbf0 con 0x7fc0e013f480 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.227+0000 7fc0de800640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40039b0 0x7fc0c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0de800640 1 -- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 msgr2=0x7fc0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0de800640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e013f480 0x7fc0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0de800640 1 -- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c4004800 msgr2=0x7fc0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0dde00640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c4004800 0x7fc0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0de800640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c4004800 0x7fc0c4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.228+0000 7fc0de800640 1 -- 172.21.15.94:0/4273744466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c4003520 con 0x7fc0c40039b0 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.229+0000 7fc0de800640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40039b0 0x7fc0c4003dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000ba00 tx=0x7fc0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915a77c64f739385 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.229+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0030070 con 0x7fc0c40039b0 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.229+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d002adb0 con 0x7fc0c40039b0 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.229+0000 7fc0dd400640 1 -- 172.21.15.94:0/4273744466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000b600 con 0x7fc0c40039b0 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 -- 172.21.15.94:0/4273744466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40039b0 msgr2=0x7fc0c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/4273744466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40039b0 0x7fc0c4003dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000ba00 tx=0x7fc0c4005830 comp rx=0 tx=0).stop 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0df200640 1 -- 172.21.15.94:0/4273744466 reap_dead start 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 -- 172.21.15.94:0/4273744466 shutdown_connections 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 -- 172.21.15.94:0/4273744466 >> 172.21.15.94:0/4273744466 conn(0x7fc0e00b6230 msgr2=0x7fc0e00b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 -- 172.21.15.94:0/4273744466 shutdown_connections 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.230+0000 7fc0e4a00640 1 -- 172.21.15.94:0/4273744466 wait complete. 2024-05-27T12:35:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.231+0000 7fc0e4a00640 1 Processor -- start 2024-05-27T12:35:32.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.232+0000 7fc0e4a00640 1 -- start start 2024-05-27T12:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.232+0000 7fc0e4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 0x7fc0e01561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.232+0000 7fc0e4a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0133780 0x7fc0e0133c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0e4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0e4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0140200 con 0x7fc0e0159340 2024-05-27T12:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0e4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0140570 con 0x7fc0c40039b0 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0e4a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e013f860 con 0x7fc0e0133780 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56730/0 (socket says 172.21.15.94:56730) 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 -- 172.21.15.94:0/309036559 learned_addr learned my addr 172.21.15.94:0/309036559 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0de800640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 0x7fc0e01561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 -- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0133780 msgr2=0x7fc0e0133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0133780 0x7fc0e0133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 -- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 msgr2=0x7fc0e01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 0x7fc0e01561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0df200640 1 -- 172.21.15.94:0/309036559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e013e690 con 0x7fc0e0159340 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.233+0000 7fc0de800640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 0x7fc0e01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0df200640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0e015d6d0 tx=0x7fc0d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800ed90 con 0x7fc0e0159340 2024-05-27T12:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d8019070 con 0x7fc0e0159340 2024-05-27T12:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e01582a0 con 0x7fc0e0159340 2024-05-27T12:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8014660 con 0x7fc0e0159340 2024-05-27T12:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.234+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e0158860 con 0x7fc0e0159340 2024-05-27T12:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.235+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e0075460 con 0x7fc0e0159340 2024-05-27T12:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.236+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0d8014800 con 0x7fc0e0159340 2024-05-27T12:35:32.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.237+0000 7fc0d6a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 0x7fc0bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.238+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0d808bed0 con 0x7fc0e0159340 2024-05-27T12:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.238+0000 7fc0de800640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 0x7fc0bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.239+0000 7fc0de800640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 0x7fc0bc06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40057e0 tx=0x7fc0d0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.242+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0e013e690 con 0x7fc0e0159340 2024-05-27T12:35:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.424+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0e0072f30 con 0x7fc0e0159340 2024-05-27T12:35:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.425+0000 7fc0d6a00640 1 -- 172.21.15.94:0/309036559 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc0d805a2f0 con 0x7fc0e0159340 2024-05-27T12:35:32.424 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.427+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 msgr2=0x7fc0bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.427+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 0x7fc0bc06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40057e0 tx=0x7fc0d0030000 comp rx=0 tx=0).stop 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.427+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 msgr2=0x7fc0e01597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.427+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0e015d6d0 tx=0x7fc0d800e4b0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 shutdown_connections 2024-05-27T12:35:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0bc06c8f0 0x7fc0bc06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0159340 0x7fc0e01597c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0133780 0x7fc0e0133c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 --2- 172.21.15.94:0/309036559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c40039b0 0x7fc0e01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 >> 172.21.15.94:0/309036559 conn(0x7fc0e00b6230 msgr2=0x7fc0e015aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 shutdown_connections 2024-05-27T12:35:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.428+0000 7fc0e4a00640 1 -- 172.21.15.94:0/309036559 wait complete. 2024-05-27T12:35:32.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:32.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.640+0000 7fe3c1400640 1 Processor -- start 2024-05-27T12:35:32.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.640+0000 7fe3c1400640 1 -- start start 2024-05-27T12:35:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3c1400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0cfb40 0x7fe3bc0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3c1400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3c1400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3bc15eee0 con 0x7fe3bc15eb00 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3c1400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe3bc0d5700 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3c1400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d21a0 con 0x7fe3bc0cfb40 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3bb200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3bb200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56742/0 (socket says 172.21.15.94:56742) 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.642+0000 7fe3bb200640 1 -- 172.21.15.94:0/2610800673 learned_addr learned my addr 172.21.15.94:0/2610800673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.643+0000 7fe3bb200640 1 -- 172.21.15.94:0/2610800673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0cfb40 msgr2=0x7fe3bc0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.643+0000 7fe3bb200640 1 --2- 172.21.15.94:0/2610800673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0cfb40 0x7fe3bc0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.643+0000 7fe3bb200640 1 -- 172.21.15.94:0/2610800673 >> v1:172.21.15.97:6789/0 conn(0x7fe3bc15eb00 legacy=0x7fe3bc0d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.643+0000 7fe3bb200640 1 -- 172.21.15.94:0/2610800673 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc0d4660 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3bb200640 1 --2- 172.21.15.94:0/2610800673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009e80 tx=0x7fe3b402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c65303c7cf04c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3b9400640 1 -- 172.21.15.94:0/2610800673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036070 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3b9400640 1 -- 172.21.15.94:0/2610800673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b4031600 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3b9400640 1 -- 172.21.15.94:0/2610800673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40319d0 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3c1400640 1 -- 172.21.15.94:0/2610800673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0d1d70 msgr2=0x7fe3bc0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3c1400640 1 --2- 172.21.15.94:0/2610800673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009e80 tx=0x7fe3b402a780 comp rx=0 tx=0).stop 2024-05-27T12:35:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.644+0000 7fe3c1400640 1 -- 172.21.15.94:0/2610800673 shutdown_connections 2024-05-27T12:35:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.645+0000 7fe3c1400640 1 --2- 172.21.15.94:0/2610800673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0d1d70 0x7fe3bc0b78f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.645+0000 7fe3c1400640 1 --2- 172.21.15.94:0/2610800673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0cfb40 0x7fe3bc0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.645+0000 7fe3c1400640 1 -- 172.21.15.94:0/2610800673 >> 172.21.15.94:0/2610800673 conn(0x7fe3bc067a50 msgr2=0x7fe3bc067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.645+0000 7fe3c1400640 1 -- 172.21.15.94:0/2610800673 shutdown_connections 2024-05-27T12:35:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.645+0000 7fe3c1400640 1 -- 172.21.15.94:0/2610800673 wait complete. 2024-05-27T12:35:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.646+0000 7fe3c1400640 1 Processor -- start 2024-05-27T12:35:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.646+0000 7fe3c1400640 1 -- start start 2024-05-27T12:35:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 0x7fe3bc0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0d1d70 0x7fe3bc0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d4f00 con 0x7fe3bc15eb00 2024-05-27T12:35:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d4c20 con 0x7fe3bc0cfb40 2024-05-27T12:35:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.647+0000 7fe3c1400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d51e0 con 0x7fe3bc0d1d70 2024-05-27T12:35:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56754/0 (socket says 172.21.15.94:56754) 2024-05-27T12:35:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 -- 172.21.15.94:0/376974987 learned_addr learned my addr 172.21.15.94:0/376974987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3ba800640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 0x7fe3bc0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3b9e00640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0d1d70 0x7fe3bc0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 -- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0d1d70 msgr2=0x7fe3bc0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0d1d70 0x7fe3bc0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 -- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 msgr2=0x7fe3bc0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 0x7fe3bc0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3bb200640 1 -- 172.21.15.94:0/376974987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc0c85a0 con 0x7fe3bc15eb00 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.648+0000 7fe3ba800640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 0x7fe3bc0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3bb200640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40093e0 tx=0x7fe3b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036040 con 0x7fe3bc15eb00 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b4003990 con 0x7fe3bc15eb00 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4009510 con 0x7fe3bc15eb00 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc0c8880 con 0x7fe3bc15eb00 2024-05-27T12:35:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.649+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc0c8e40 con 0x7fe3bc15eb00 2024-05-27T12:35:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.650+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc0c9280 con 0x7fe3bc15eb00 2024-05-27T12:35:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.652+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3b4007780 con 0x7fe3bc15eb00 2024-05-27T12:35:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.652+0000 7fe3b2a00640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.653+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3b40ac9d0 con 0x7fe3bc15eb00 2024-05-27T12:35:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.653+0000 7fe3ba800640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.654+0000 7fe3ba800640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00a810 tx=0x7fe3ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:32.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.657+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3b407aec0 con 0x7fe3bc15eb00 2024-05-27T12:35:32.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.855+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3bc0c9280 con 0x7fe3bc15eb00 2024-05-27T12:35:32.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.856+0000 7fe3b2a00640 1 -- 172.21.15.94:0/376974987 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe3b407a7c0 con 0x7fe3bc15eb00 2024-05-27T12:35:32.856 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 msgr2=0x7fe39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00a810 tx=0x7fe3ac008040 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 msgr2=0x7fe3bc15fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40093e0 tx=0x7fe3b4004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 shutdown_connections 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc15eb00 0x7fe3bc15fae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0d1d70 0x7fe3bc0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 --2- 172.21.15.94:0/376974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0cfb40 0x7fe3bc0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 >> 172.21.15.94:0/376974987 conn(0x7fe3bc067a50 msgr2=0x7fe3bc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 shutdown_connections 2024-05-27T12:35:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:32.859+0000 7fe3c1400640 1 -- 172.21.15.94:0/376974987 wait complete. 2024-05-27T12:35:32.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.084+0000 7faaf8400640 1 Processor -- start 2024-05-27T12:35:33.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.084+0000 7faaf8400640 1 -- start start 2024-05-27T12:35:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.086+0000 7faaf8400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faaf0146060 0x7faaf00b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.086+0000 7faaf8400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.086+0000 7faaf8400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf015cfd0 con 0x7faaf00cd8a0 2024-05-27T12:35:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.086+0000 7faaf8400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7faaf015dbe0 con 0x7faaf0146060 2024-05-27T12:35:33.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.086+0000 7faaf8400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faaf015d2b0 con 0x7faaf006c040 2024-05-27T12:35:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.087+0000 7faaf5a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faaf0146060 0x7faaf00b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.087+0000 7faaf6400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.087+0000 7faaf6400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:41544/0 (socket says 172.21.15.94:41544) 2024-05-27T12:35:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.087+0000 7faaf6400640 1 -- 172.21.15.94:0/3290734475 learned_addr learned my addr 172.21.15.94:0/3290734475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.088+0000 7faaf6400640 1 -- 172.21.15.94:0/3290734475 >> v2:172.21.15.97:3300/0 conn(0x7faaf0146060 msgr2=0x7faaf00b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.088+0000 7faaf6400640 1 --2- 172.21.15.94:0/3290734475 >> v2:172.21.15.97:3300/0 conn(0x7faaf0146060 0x7faaf00b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.088+0000 7faaf6400640 1 -- 172.21.15.94:0/3290734475 >> v1:172.21.15.97:6789/0 conn(0x7faaf00cd8a0 legacy=0x7faaf006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.088+0000 7faaf6400640 1 -- 172.21.15.94:0/3290734475 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf015d590 con 0x7faaf006c040 2024-05-27T12:35:33.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.089+0000 7faaf6400640 1 --2- 172.21.15.94:0/3290734475 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2c14d8e76cd82b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.089+0000 7faaefe00640 1 -- 172.21.15.94:0/3290734475 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8036070 con 0x7faaf006c040 2024-05-27T12:35:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.089+0000 7faaefe00640 1 -- 172.21.15.94:0/3290734475 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae8031600 con 0x7faaf006c040 2024-05-27T12:35:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.089+0000 7faaefe00640 1 -- 172.21.15.94:0/3290734475 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae80319d0 con 0x7faaf006c040 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.090+0000 7faaf8400640 1 -- 172.21.15.94:0/3290734475 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 msgr2=0x7faaf0072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.090+0000 7faaf8400640 1 --2- 172.21.15.94:0/3290734475 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae802a710 comp rx=0 tx=0).stop 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 -- 172.21.15.94:0/3290734475 shutdown_connections 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 --2- 172.21.15.94:0/3290734475 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf006c040 0x7faaf0072010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 --2- 172.21.15.94:0/3290734475 >> v2:172.21.15.97:3300/0 conn(0x7faaf0146060 0x7faaf00b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 -- 172.21.15.94:0/3290734475 >> 172.21.15.94:0/3290734475 conn(0x7faaf00b2240 msgr2=0x7faaf00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 -- 172.21.15.94:0/3290734475 shutdown_connections 2024-05-27T12:35:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.091+0000 7faaf8400640 1 -- 172.21.15.94:0/3290734475 wait complete. 2024-05-27T12:35:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.093+0000 7faaf8400640 1 Processor -- start 2024-05-27T12:35:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.093+0000 7faaf8400640 1 -- start start 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaf00cd8a0 0x7faaf013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf0146060 0x7faaf013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaf00d0110 con 0x7faaf006c040 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0146440 con 0x7faaf00cd8a0 2024-05-27T12:35:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf8400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faaf00cdd30 con 0x7faaf0146060 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.094+0000 7faaf5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56770/0 (socket says 172.21.15.94:56770) 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 -- 172.21.15.94:0/4094389498 learned_addr learned my addr 172.21.15.94:0/4094389498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf6400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf0146060 0x7faaf013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5000640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaf00cd8a0 0x7faaf013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 -- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf0146060 msgr2=0x7faaf013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf0146060 0x7faaf013f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 -- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaf00cd8a0 msgr2=0x7faaf013ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaf00cd8a0 0x7faaf013ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 -- 172.21.15.94:0/4094389498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf013f1f0 con 0x7faaf006c040 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.095+0000 7faaf5a00640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faae400e8a0 tx=0x7faae400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.096+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4017070 con 0x7faaf006c040 2024-05-27T12:35:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.096+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae400c940 con 0x7faaf006c040 2024-05-27T12:35:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.096+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400cd30 con 0x7faaf006c040 2024-05-27T12:35:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.096+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf006f520 con 0x7faaf006c040 2024-05-27T12:35:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.096+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf006fae0 con 0x7faaf006c040 2024-05-27T12:35:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.097+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0068a70 con 0x7faaf006c040 2024-05-27T12:35:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.098+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faae401a070 con 0x7faaf006c040 2024-05-27T12:35:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.099+0000 7faaed600640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.099+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faae408ba40 con 0x7faaf006c040 2024-05-27T12:35:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.100+0000 7faaf5000640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.101+0000 7faaf5000640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faaf00aecc0 tx=0x7faadc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.104+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faaf013f1f0 con 0x7faaf006c040 2024-05-27T12:35:33.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.300+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faaf006dd60 con 0x7faaf006c040 2024-05-27T12:35:33.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.301+0000 7faaed600640 1 -- 172.21.15.94:0/4094389498 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7faae4059dd0 con 0x7faaf006c040 2024-05-27T12:35:33.301 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 msgr2=0x7faad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faaf00aecc0 tx=0x7faadc009040 comp rx=0 tx=0).stop 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 msgr2=0x7faaf015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faae400e8a0 tx=0x7faae400ed60 comp rx=0 tx=0).stop 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 shutdown_connections 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaf00cd8a0 0x7faaf013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaf0146060 0x7faaf013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 --2- 172.21.15.94:0/4094389498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf006c040 0x7faaf015b980 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faae400e8a0 tx=0x7faae400ed60 comp rx=0 tx=0).stop 2024-05-27T12:35:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 >> 172.21.15.94:0/4094389498 conn(0x7faaf00b2240 msgr2=0x7faaf0142ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.303+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 shutdown_connections 2024-05-27T12:35:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.304+0000 7faaf8400640 1 -- 172.21.15.94:0/4094389498 wait complete. 2024-05-27T12:35:33.323 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:33.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.524+0000 7fcf95200640 1 Processor -- start 2024-05-27T12:35:33.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.524+0000 7fcf95200640 1 -- start start 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcf9015ac60 0x7fcf9006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf900cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcf900cfcd0 0x7fcf900725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcf900cf300 con 0x7fcf900cfcd0 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9006f860 con 0x7fcf9015ac60 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.525+0000 7fcf95200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcf900cecb0 con 0x7fcf900b3e20 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcf900cfcd0 0x7fcf900725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcf900cfcd0 0x7fcf900725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56778/0 (socket says 172.21.15.94:56778) 2024-05-27T12:35:33.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 -- 172.21.15.94:0/4110100036 learned_addr learned my addr 172.21.15.94:0/4110100036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8ea00640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf900cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8f400640 1 --2- 172.21.15.94:0/4110100036 >> v2:172.21.15.97:3300/0 conn(0x7fcf9015ac60 0x7fcf9006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 -- 172.21.15.94:0/4110100036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 msgr2=0x7fcf900cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf900cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 -- 172.21.15.94:0/4110100036 >> v2:172.21.15.97:3300/0 conn(0x7fcf9015ac60 msgr2=0x7fcf9006f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 --2- 172.21.15.94:0/4110100036 >> v2:172.21.15.97:3300/0 conn(0x7fcf9015ac60 0x7fcf9006f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.526+0000 7fcf8fe00640 1 -- 172.21.15.94:0/4110100036 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf900cef90 con 0x7fcf900cfcd0 2024-05-27T12:35:33.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.528+0000 7fcf8fe00640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf900725f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf90073180 tx=0x7fcf8802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a2f6d17b6a3c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.528+0000 7fcf8e000640 1 -- 172.21.15.94:0/4110100036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88036070 con 0x7fcf900cfcd0 2024-05-27T12:35:33.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.528+0000 7fcf8e000640 1 -- 172.21.15.94:0/4110100036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf88031600 con 0x7fcf900cfcd0 2024-05-27T12:35:33.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.528+0000 7fcf8e000640 1 -- 172.21.15.94:0/4110100036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880319d0 con 0x7fcf900cfcd0 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 -- 172.21.15.94:0/4110100036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 msgr2=0x7fcf900725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf900725f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf90073180 tx=0x7fcf8802a610 comp rx=0 tx=0).stop 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 -- 172.21.15.94:0/4110100036 shutdown_connections 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf900725f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 --2- 172.21.15.94:0/4110100036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf900cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 --2- 172.21.15.94:0/4110100036 >> v2:172.21.15.97:3300/0 conn(0x7fcf9015ac60 0x7fcf9006f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 -- 172.21.15.94:0/4110100036 >> 172.21.15.94:0/4110100036 conn(0x7fcf900b2160 msgr2=0x7fcf900b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 -- 172.21.15.94:0/4110100036 shutdown_connections 2024-05-27T12:35:33.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.529+0000 7fcf95200640 1 -- 172.21.15.94:0/4110100036 wait complete. 2024-05-27T12:35:33.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.530+0000 7fcf95200640 1 Processor -- start 2024-05-27T12:35:33.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.531+0000 7fcf95200640 1 -- start start 2024-05-27T12:35:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf90142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf9015ac60 0x7fcf90152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90072b30 con 0x7fcf900cfcd0 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf900b4280 con 0x7fcf9015ac60 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf95200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcf90072cb0 con 0x7fcf900b3e20 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56780/0 (socket says 172.21.15.94:56780) 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 -- 172.21.15.94:0/2736260137 learned_addr learned my addr 172.21.15.94:0/2736260137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8f400640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf90142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8fe00640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf9015ac60 0x7fcf90152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 -- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 msgr2=0x7fcf90142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf90142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 -- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf9015ac60 msgr2=0x7fcf90152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf9015ac60 0x7fcf90152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.532+0000 7fcf8ea00640 1 -- 172.21.15.94:0/2736260137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9015c750 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf8ea00640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00d800 tx=0x7fcf7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c018020 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c0096a0 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c009a70 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9015ca30 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.533+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9014a340 con 0x7fcf900cfcd0 2024-05-27T12:35:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.534+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf90075460 con 0x7fcf900cfcd0 2024-05-27T12:35:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.535+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf7c01b020 con 0x7fcf900cfcd0 2024-05-27T12:35:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.536+0000 7fcf87400640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.536+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c08c120 con 0x7fcf900cfcd0 2024-05-27T12:35:33.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.537+0000 7fcf8f400640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.538+0000 7fcf8f400640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 0x7fcf7006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcf80006fd0 tx=0x7fcf80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.540+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf7c022020 con 0x7fcf900cfcd0 2024-05-27T12:35:33.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.745+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf90151690 con 0x7fcf900cfcd0 2024-05-27T12:35:33.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.746+0000 7fcf87400640 1 -- 172.21.15.94:0/2736260137 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fcf7c05a4b0 con 0x7fcf900cfcd0 2024-05-27T12:35:33.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:33.746 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 msgr2=0x7fcf7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 0x7fcf7006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcf80006fd0 tx=0x7fcf80008040 comp rx=0 tx=0).stop 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 msgr2=0x7fcf9015d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00d800 tx=0x7fcf7c00dcc0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 shutdown_connections 2024-05-27T12:35:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcf7006c8f0 0x7fcf7006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf9015ac60 0x7fcf90152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf900cfcd0 0x7fcf9015d410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 --2- 172.21.15.94:0/2736260137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf900b3e20 0x7fcf90142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 >> 172.21.15.94:0/2736260137 conn(0x7fcf900b2160 msgr2=0x7fcf90159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 shutdown_connections 2024-05-27T12:35:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.749+0000 7fcf95200640 1 -- 172.21.15.94:0/2736260137 wait complete. 2024-05-27T12:35:33.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:33.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:33.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.975+0000 7fae09c00640 1 Processor -- start 2024-05-27T12:35:33.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.975+0000 7fae09c00640 1 -- start start 2024-05-27T12:35:33.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.977+0000 7fae09c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae04146060 0x7fae040b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.977+0000 7fae09c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cd8a0 0x7fae0406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.977+0000 7fae09c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0415cfd0 con 0x7fae040cdd00 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.977+0000 7fae09c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fae0415dbe0 con 0x7fae04146060 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.977+0000 7fae09c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fae0415d2b0 con 0x7fae040cd8a0 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.978+0000 7fae02a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae04146060 0x7fae040b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.978+0000 7fae02000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cd8a0 0x7fae0406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.978+0000 7fae02a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae04146060 0x7fae040b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:51812/0 (socket says 172.21.15.94:51812) 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.978+0000 7fae02a00640 1 -- 172.21.15.94:0/2015491711 learned_addr learned my addr 172.21.15.94:0/2015491711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.979+0000 7fae02a00640 1 -- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cd8a0 msgr2=0x7fae0406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.979+0000 7fae02a00640 1 --2- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cd8a0 0x7fae0406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.979+0000 7fae02a00640 1 -- 172.21.15.94:0/2015491711 >> v1:172.21.15.97:6789/0 conn(0x7fae040cdd00 legacy=0x7fae04072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.979+0000 7fae02a00640 1 -- 172.21.15.94:0/2015491711 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0415d590 con 0x7fae04146060 2024-05-27T12:35:33.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.980+0000 7fae02a00640 1 --2- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae04146060 0x7fae040b3f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009e50 tx=0x7fadf402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d71274689d02662 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.980+0000 7fae01600640 1 -- 172.21.15.94:0/2015491711 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4036070 con 0x7fae04146060 2024-05-27T12:35:33.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.980+0000 7fae01600640 1 -- 172.21.15.94:0/2015491711 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf4031600 con 0x7fae04146060 2024-05-27T12:35:33.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.980+0000 7fae01600640 1 -- 172.21.15.94:0/2015491711 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf40319d0 con 0x7fae04146060 2024-05-27T12:35:33.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 -- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae04146060 msgr2=0x7fae040b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 --2- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae04146060 0x7fae040b3f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009e50 tx=0x7fadf402a710 comp rx=0 tx=0).stop 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 -- 172.21.15.94:0/2015491711 shutdown_connections 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 --2- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cd8a0 0x7fae0406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 --2- 172.21.15.94:0/2015491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae04146060 0x7fae040b3f70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 -- 172.21.15.94:0/2015491711 >> 172.21.15.94:0/2015491711 conn(0x7fae040b2240 msgr2=0x7fae040b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 -- 172.21.15.94:0/2015491711 shutdown_connections 2024-05-27T12:35:33.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.981+0000 7fae09c00640 1 -- 172.21.15.94:0/2015491711 wait complete. 2024-05-27T12:35:33.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.983+0000 7fae09c00640 1 Processor -- start 2024-05-27T12:35:33.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.983+0000 7fae09c00640 1 -- start start 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae040cd8a0 0x7fae0415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 0x7fae04144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae040d0110 con 0x7fae04146060 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae04146440 con 0x7fae040cd8a0 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae09c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fae0406be70 con 0x7fae040cdd00 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae03400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae03400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56794/0 (socket says 172.21.15.94:56794) 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.984+0000 7fae03400640 1 -- 172.21.15.94:0/296227902 learned_addr learned my addr 172.21.15.94:0/296227902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:33.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae02a00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae040cd8a0 0x7fae0415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae02000640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 0x7fae04144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 -- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 msgr2=0x7fae04144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 0x7fae04144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 -- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae040cd8a0 msgr2=0x7fae0415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae040cd8a0 0x7fae0415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 -- 172.21.15.94:0/296227902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0415c860 con 0x7fae04146060 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae02000640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 0x7fae04144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae03400640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fadfc001eb0 tx=0x7fadfc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc017070 con 0x7fae04146060 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc0052d0 con 0x7fae04146060 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc003810 con 0x7fae04146060 2024-05-27T12:35:33.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae040ae2b0 con 0x7fae04146060 2024-05-27T12:35:33.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.985+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae040ae830 con 0x7fae04146060 2024-05-27T12:35:33.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.987+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae04068a70 con 0x7fae04146060 2024-05-27T12:35:33.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.987+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadfc01a070 con 0x7fae04146060 2024-05-27T12:35:33.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.988+0000 7fadfaa00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 0x7fade406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:33.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.988+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fadfc08af50 con 0x7fae04146060 2024-05-27T12:35:33.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.989+0000 7fae02a00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 0x7fade406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:33.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.990+0000 7fae02a00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 0x7fade406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadf4033000 tx=0x7fadf4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:33.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:33.993+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadfc0592e0 con 0x7fae04146060 2024-05-27T12:35:34.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.196+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae040b3840 con 0x7fae04146060 2024-05-27T12:35:34.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.197+0000 7fadfaa00640 1 -- 172.21.15.94:0/296227902 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fadfc058be0 con 0x7fae04146060 2024-05-27T12:35:34.196 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:34.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.199+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 msgr2=0x7fade406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.199+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 0x7fade406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadf4033000 tx=0x7fadf4009000 comp rx=0 tx=0).stop 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.199+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 msgr2=0x7fae04145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.199+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fadfc001eb0 tx=0x7fadfc00c890 comp rx=0 tx=0).stop 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 shutdown_connections 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fade406c920 0x7fade406ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04146060 0x7fae04145040 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae040cdd00 0x7fae04144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 --2- 172.21.15.94:0/296227902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae040cd8a0 0x7fae0415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 >> 172.21.15.94:0/296227902 conn(0x7fae040b2240 msgr2=0x7fae040b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:34.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 shutdown_connections 2024-05-27T12:35:34.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.200+0000 7fae09c00640 1 -- 172.21.15.94:0/296227902 wait complete. 2024-05-27T12:35:34.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:34.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.417+0000 7faa42400640 1 Processor -- start 2024-05-27T12:35:34.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.417+0000 7faa42400640 1 -- start start 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.419+0000 7faa42400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa3c072570 0x7faa3c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.419+0000 7faa42400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.419+0000 7faa42400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa3c0b4be0 con 0x7faa3c15bdc0 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.419+0000 7faa42400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7faa3c06e1d0 con 0x7faa3c072570 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.419+0000 7faa42400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faa3c0b4ea0 con 0x7faa3c0b4450 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa3c072570 0x7faa3c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa3be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa3c072570 0x7faa3c0b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:51828/0 (socket says 172.21.15.94:51828) 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 -- 172.21.15.94:0/3954030843 learned_addr learned my addr 172.21.15.94:0/3954030843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 -- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 msgr2=0x7faa3c06e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 --2- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c06e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 -- 172.21.15.94:0/3954030843 >> v1:172.21.15.97:6789/0 conn(0x7faa3c15bdc0 legacy=0x7faa3c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.420+0000 7faa32a00640 1 -- 172.21.15.94:0/3954030843 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c0b5180 con 0x7faa3c072570 2024-05-27T12:35:34.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.422+0000 7faa32a00640 1 --2- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c0b3f10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faa24009870 tx=0x7faa2402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3783ec958e339ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.422+0000 7faa3aa00640 1 -- 172.21.15.94:0/3954030843 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24037070 con 0x7faa3c072570 2024-05-27T12:35:34.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.422+0000 7faa3aa00640 1 -- 172.21.15.94:0/3954030843 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa24032440 con 0x7faa3c072570 2024-05-27T12:35:34.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.422+0000 7faa3aa00640 1 -- 172.21.15.94:0/3954030843 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24032810 con 0x7faa3c072570 2024-05-27T12:35:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 -- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 msgr2=0x7faa3c0b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 --2- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c0b3f10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faa24009870 tx=0x7faa2402a270 comp rx=0 tx=0).stop 2024-05-27T12:35:34.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 -- 172.21.15.94:0/3954030843 shutdown_connections 2024-05-27T12:35:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 --2- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c06e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 --2- 172.21.15.94:0/3954030843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c0b3f10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 -- 172.21.15.94:0/3954030843 >> 172.21.15.94:0/3954030843 conn(0x7faa3c0b2240 msgr2=0x7faa3c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 -- 172.21.15.94:0/3954030843 shutdown_connections 2024-05-27T12:35:34.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.423+0000 7faa42400640 1 -- 172.21.15.94:0/3954030843 wait complete. 2024-05-27T12:35:34.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.424+0000 7faa42400640 1 Processor -- start 2024-05-27T12:35:34.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.425+0000 7faa42400640 1 -- start start 2024-05-27T12:35:34.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.425+0000 7faa42400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.425+0000 7faa42400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c168d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa42400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa42400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c15c1a0 con 0x7faa3c15bdc0 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa42400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c0b4880 con 0x7faa3c072570 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa42400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faa3c15c320 con 0x7faa3c0b4450 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56796/0 (socket says 172.21.15.94:56796) 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 -- 172.21.15.94:0/3875942958 learned_addr learned my addr 172.21.15.94:0/3875942958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa32a00640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c168d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3b400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 -- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 msgr2=0x7faa3c168d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c168d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 -- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 msgr2=0x7faa3c14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.426+0000 7faa3be00640 1 -- 172.21.15.94:0/3875942958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c167fd0 con 0x7faa3c15bdc0 2024-05-27T12:35:34.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa3be00640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7f0 tx=0x7faa3400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34017070 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3400ee40 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400cb80 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c168290 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.427+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c14fca0 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.428+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3c068a70 con 0x7faa3c15bdc0 2024-05-27T12:35:34.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.429+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3401a070 con 0x7faa3c15bdc0 2024-05-27T12:35:34.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.430+0000 7faa33e00640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 0x7faa1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.430+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa3408ba00 con 0x7faa3c15bdc0 2024-05-27T12:35:34.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.431+0000 7faa3b400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 0x7faa1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.432+0000 7faa3b400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 0x7faa1806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001d40 tx=0x7faa2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.437+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa34059e20 con 0x7faa3c15bdc0 2024-05-27T12:35:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.641+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa3c0730c0 con 0x7faa3c15bdc0 2024-05-27T12:35:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.642+0000 7faa33e00640 1 -- 172.21.15.94:0/3875942958 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7faa34059720 con 0x7faa3c15bdc0 2024-05-27T12:35:34.642 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:34.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.645+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 msgr2=0x7faa1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.645+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 0x7faa1806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001d40 tx=0x7faa2c009040 comp rx=0 tx=0).stop 2024-05-27T12:35:34.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.645+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 msgr2=0x7faa3c14f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.645+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7f0 tx=0x7faa3400ebf0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 shutdown_connections 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faa1806c8f0 0x7faa1806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa3c15bdc0 0x7faa3c14f760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa3c0b4450 0x7faa3c168d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 --2- 172.21.15.94:0/3875942958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c072570 0x7faa3c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 >> 172.21.15.94:0/3875942958 conn(0x7faa3c0b2240 msgr2=0x7faa3c06f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 shutdown_connections 2024-05-27T12:35:34.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.646+0000 7faa42400640 1 -- 172.21.15.94:0/3875942958 wait complete. 2024-05-27T12:35:34.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.869+0000 7ffb65c00640 1 Processor -- start 2024-05-27T12:35:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.870+0000 7ffb65c00640 1 -- start start 2024-05-27T12:35:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb65c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb6009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb65c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb600d2020 con 0x7ffb6009af70 2024-05-27T12:35:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb65c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb600d1d40 con 0x7ffb60162150 2024-05-27T12:35:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb65c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6009b3c0 con 0x7ffb6006ca40 2024-05-27T12:35:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb5f400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ffb6009af70 0x7ffb600cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54508/0 (socket says 172.21.15.94:54508) 2024-05-27T12:35:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.871+0000 7ffb5f400640 1 -- 172.21.15.94:0/3593305862 learned_addr learned my addr 172.21.15.94:0/3593305862 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5e000640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb6009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921199916 0 0) 0x7ffb600d2020 con 0x7ffb6009af70 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb44003520 con 0x7ffb6009af70 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1955181576 0 0) 0x7ffb44003520 con 0x7ffb6009af70 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 msgr2=0x7ffb6009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb6009aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 >> v1:172.21.15.97:6789/0 conn(0x7ffb60162150 legacy=0x7ffb600d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb600ccd60 con 0x7ffb6009af70 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7ffb580019f0 con 0x7ffb6009af70 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 >> v1:172.21.15.94:6789/0 conn(0x7ffb6009af70 legacy=0x7ffb600cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.872+0000 7ffb5d600640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb44003fb0 0x7ffb44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5d600640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44004d40 0x7ffb440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6009b3c0 con 0x7ffb44003fb0 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4400bb00 con 0x7ffb44004d40 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4400bd00 con 0x7ffb6006ca40 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb44003fb0 0x7ffb44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5f400640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44004d40 0x7ffb440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5ea00640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 -- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 msgr2=0x7ffb440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb6006ca40 0x7ffb440039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 -- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44004d40 msgr2=0x7ffb440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44004d40 0x7ffb440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5e000640 1 -- 172.21.15.94:0/3593305862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb44003520 con 0x7ffb44003fb0 2024-05-27T12:35:34.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.873+0000 7ffb5f400640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44004d40 0x7ffb440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:34.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.874+0000 7ffb5e000640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb44003fb0 0x7ffb44004800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009ac0 tx=0x7ffb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39a007a38c69eec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48036070 con 0x7ffb44003fb0 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48031400 con 0x7ffb44003fb0 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb5d600640 1 -- 172.21.15.94:0/3593305862 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48009690 con 0x7ffb44003fb0 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 -- 172.21.15.94:0/3593305862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb44003fb0 msgr2=0x7ffb44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 --2- 172.21.15.94:0/3593305862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb44003fb0 0x7ffb44004800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009ac0 tx=0x7ffb44005830 comp rx=0 tx=0).stop 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb5f400640 1 -- 172.21.15.94:0/3593305862 reap_dead start 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 -- 172.21.15.94:0/3593305862 shutdown_connections 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 -- 172.21.15.94:0/3593305862 >> 172.21.15.94:0/3593305862 conn(0x7ffb600b4160 msgr2=0x7ffb600b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 -- 172.21.15.94:0/3593305862 shutdown_connections 2024-05-27T12:35:34.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.875+0000 7ffb65c00640 1 -- 172.21.15.94:0/3593305862 wait complete. 2024-05-27T12:35:34.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.877+0000 7ffb65c00640 1 Processor -- start 2024-05-27T12:35:34.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.877+0000 7ffb65c00640 1 -- start start 2024-05-27T12:35:34.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44003fb0 0x7ffb600c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb600b7ec0 0x7ffb600c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6009aa30 con 0x7ffb600b7a40 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600cd320 con 0x7ffb44003fb0 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb65c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6009abb0 con 0x7ffb600b7ec0 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56822/0 (socket says 172.21.15.94:56822) 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.878+0000 7ffb5e000640 1 -- 172.21.15.94:0/2818731952 learned_addr learned my addr 172.21.15.94:0/2818731952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5ea00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44003fb0 0x7ffb600c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5f400640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb600b7ec0 0x7ffb600c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 -- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb600b7ec0 msgr2=0x7ffb600c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb600b7ec0 0x7ffb600c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 -- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44003fb0 msgr2=0x7ffb600c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44003fb0 0x7ffb600c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 -- 172.21.15.94:0/2818731952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb600c68a0 con 0x7ffb600b7a40 2024-05-27T12:35:34.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb5e000640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.879+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48040020 con 0x7ffb600b7a40 2024-05-27T12:35:34.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.880+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48003c60 con 0x7ffb600b7a40 2024-05-27T12:35:34.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.880+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48036040 con 0x7ffb600b7a40 2024-05-27T12:35:34.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.880+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb600afdb0 con 0x7ffb600b7a40 2024-05-27T12:35:34.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.880+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb600b0370 con 0x7ffb600b7a40 2024-05-27T12:35:34.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.881+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb60068a70 con 0x7ffb600b7a40 2024-05-27T12:35:34.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.882+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb48043020 con 0x7ffb600b7a40 2024-05-27T12:35:34.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.883+0000 7ffb56a00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:34.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.883+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb480ac140 con 0x7ffb600b7a40 2024-05-27T12:35:34.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.884+0000 7ffb5ea00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:34.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.885+0000 7ffb5ea00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001690 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:34.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:34.888+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb4804a030 con 0x7ffb600b7a40 2024-05-27T12:35:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.085+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb6009bb80 con 0x7ffb600b7a40 2024-05-27T12:35:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.086+0000 7ffb56a00640 1 -- 172.21.15.94:0/2818731952 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ffb480044e0 con 0x7ffb600b7a40 2024-05-27T12:35:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.088+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 msgr2=0x7ffb3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.088+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001690 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-05-27T12:35:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.088+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 msgr2=0x7ffb600c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.088+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48004730 comp rx=0 tx=0).stop 2024-05-27T12:35:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 shutdown_connections 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb600b7ec0 0x7ffb600c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb600b7a40 0x7ffb600c7e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 --2- 172.21.15.94:0/2818731952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb44003fb0 0x7ffb600c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 >> 172.21.15.94:0/2818731952 conn(0x7ffb600b4160 msgr2=0x7ffb60071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 shutdown_connections 2024-05-27T12:35:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.089+0000 7ffb65c00640 1 -- 172.21.15.94:0/2818731952 wait complete. 2024-05-27T12:35:35.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:35.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.301+0000 7f279f200640 1 Processor -- start 2024-05-27T12:35:35.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.301+0000 7f279f200640 1 -- start start 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.302+0000 7f279f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27a015ea30 0x7f27a00d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.302+0000 7f279f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f27a00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.302+0000 7f279f200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a015ee10 con 0x7f27a00b4510 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.302+0000 7f279f200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f27a00d2830 con 0x7f27a015ea30 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.302+0000 7f279f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00d2a60 con 0x7f27a00d4ae0 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f2795400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f27a00b4510 0x7f27a00d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54514/0 (socket says 172.21.15.94:54514) 2024-05-27T12:35:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f2795400640 1 -- 172.21.15.94:0/1288890798 learned_addr learned my addr 172.21.15.94:0/1288890798 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279de00640 1 --2- 172.21.15.94:0/1288890798 >> v2:172.21.15.97:3300/0 conn(0x7f27a015ea30 0x7f27a00d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279e800640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f27a00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788804110 0 0) 0x7f27a015ee10 con 0x7f27a00b4510 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2784003520 con 0x7f27a00b4510 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2428056895 0 0) 0x7f2784003520 con 0x7f27a00b4510 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 msgr2=0x7f27a00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f27a00cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 >> v2:172.21.15.97:3300/0 conn(0x7f27a015ea30 msgr2=0x7f27a00d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.303+0000 7f279d400640 1 --2- 172.21.15.94:0/1288890798 >> v2:172.21.15.97:3300/0 conn(0x7f27a015ea30 0x7f27a00d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a00d2d40 con 0x7f27a00b4510 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f27880017f0 con 0x7f27a00b4510 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279de00640 1 --2- 172.21.15.94:0/1288890798 >> v2:172.21.15.97:3300/0 conn(0x7f27a015ea30 0x7f27a00d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 >> v1:172.21.15.94:6789/0 conn(0x7f27a00b4510 legacy=0x7f27a00d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27840039b0 0x7f2784003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f2784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2784004d40 0x7f27840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00d2a60 con 0x7f2784004d40 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f278400bb00 con 0x7f27840039b0 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f278400bd00 con 0x7f27a00d4ae0 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279e800640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2784004d40 0x7f27840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f2795400640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f2784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279de00640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27840039b0 0x7f2784003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279e800640 1 -- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 msgr2=0x7f2784004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279e800640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a00d4ae0 0x7f2784004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279e800640 1 -- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27840039b0 msgr2=0x7f2784003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:35.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.304+0000 7f279e800640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27840039b0 0x7f2784003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.305+0000 7f279e800640 1 -- 172.21.15.94:0/1288890798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2784003520 con 0x7f2784004d40 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279e800640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2784004d40 0x7f27840051c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f279800ba30 tx=0x7f2784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fb19cc6ecc7528 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798030070 con 0x7f2784004d40 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279802ade0 con 0x7f2784004d40 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279d400640 1 -- 172.21.15.94:0/1288890798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800b600 con 0x7f2784004d40 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279f200640 1 -- 172.21.15.94:0/1288890798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2784004d40 msgr2=0x7f27840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279f200640 1 --2- 172.21.15.94:0/1288890798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2784004d40 0x7f27840051c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f279800ba30 tx=0x7f2784005830 comp rx=0 tx=0).stop 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.306+0000 7f279e800640 1 -- 172.21.15.94:0/1288890798 reap_dead start 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.307+0000 7f279f200640 1 -- 172.21.15.94:0/1288890798 shutdown_connections 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.307+0000 7f279f200640 1 -- 172.21.15.94:0/1288890798 >> 172.21.15.94:0/1288890798 conn(0x7f27a009da70 msgr2=0x7f27a008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.307+0000 7f279f200640 1 -- 172.21.15.94:0/1288890798 shutdown_connections 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.307+0000 7f279f200640 1 -- 172.21.15.94:0/1288890798 wait complete. 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.308+0000 7f279f200640 1 Processor -- start 2024-05-27T12:35:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 -- start start 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784004d40 0x7f27a00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a01f87f0 0x7f27a00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00d35e0 con 0x7f27a01f7af0 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00d3300 con 0x7f2784004d40 2024-05-27T12:35:35.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.309+0000 7f279f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00cf610 con 0x7f27a01f87f0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56846/0 (socket says 172.21.15.94:56846) 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 -- 172.21.15.94:0/2450172044 learned_addr learned my addr 172.21.15.94:0/2450172044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279de00640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784004d40 0x7f27a00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 -- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a01f87f0 msgr2=0x7f27a00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a01f87f0 0x7f27a00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 -- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784004d40 msgr2=0x7f27a00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784004d40 0x7f27a00cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 -- 172.21.15.94:0/2450172044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a00b1d40 con 0x7f27a01f7af0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.310+0000 7f279e800640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f279802a8d0 tx=0x7f27980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.311+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800b600 con 0x7f27a01f7af0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.311+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279802abe0 con 0x7f27a01f7af0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.311+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004220 con 0x7f27a01f7af0 2024-05-27T12:35:35.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.311+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a00b2020 con 0x7f27a01f7af0 2024-05-27T12:35:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.311+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a00b25e0 con 0x7f27a01f7af0 2024-05-27T12:35:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.312+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a0075460 con 0x7f27a01f7af0 2024-05-27T12:35:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.313+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2798047030 con 0x7f27a01f7af0 2024-05-27T12:35:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.314+0000 7f2796a00640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 0x7f277c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.314+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f27980abdd0 con 0x7f27a01f7af0 2024-05-27T12:35:35.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.314+0000 7f279de00640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 0x7f277c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.315+0000 7f279de00640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 0x7f277c06ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27900098c0 tx=0x7f279000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.318+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f279804e020 con 0x7f27a01f7af0 2024-05-27T12:35:35.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.502+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27a006e350 con 0x7f27a01f7af0 2024-05-27T12:35:35.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.503+0000 7f2796a00640 1 -- 172.21.15.94:0/2450172044 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f279803c3a0 con 0x7f27a01f7af0 2024-05-27T12:35:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:35.503 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:35.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 msgr2=0x7f277c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 0x7f277c06ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27900098c0 tx=0x7f279000b040 comp rx=0 tx=0).stop 2024-05-27T12:35:35.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 msgr2=0x7f27a01f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f279802a8d0 tx=0x7f27980046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 shutdown_connections 2024-05-27T12:35:35.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f277c06c920 0x7f277c06ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a01f7af0 0x7f27a01f7f70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a01f87f0 0x7f27a00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 --2- 172.21.15.94:0/2450172044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784004d40 0x7f27a00cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.506+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 >> 172.21.15.94:0/2450172044 conn(0x7f27a009da70 msgr2=0x7f27a00d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.507+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 shutdown_connections 2024-05-27T12:35:35.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.507+0000 7f279f200640 1 -- 172.21.15.94:0/2450172044 wait complete. 2024-05-27T12:35:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:35.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.728+0000 7f26ed200640 1 Processor -- start 2024-05-27T12:35:35.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.728+0000 7f26ed200640 1 -- start start 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26ed200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e813f480 0x7f26e8071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26ed200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f26e815d2f0 0x7f26e8140a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26ed200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e806dc50 con 0x7f26e8072260 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26ed200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f26e80726c0 con 0x7f26e815d2f0 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26ed200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f26e813fcd0 con 0x7f26e813f480 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26e6a00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f26e8072260 0x7f26e815cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54528/0 (socket says 172.21.15.94:54528) 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26e6a00640 1 -- 172.21.15.94:0/3436550103 learned_addr learned my addr 172.21.15.94:0/3436550103 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3436550103 >> v2:172.21.15.94:3300/0 conn(0x7f26e815d2f0 0x7f26e8140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.730+0000 7f26e7400640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e813f480 0x7f26e8071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.731+0000 7f26e7e00640 1 -- 172.21.15.94:0/3436550103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e813f480 msgr2=0x7f26e8071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.731+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e813f480 0x7f26e8071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.731+0000 7f26e7e00640 1 -- 172.21.15.94:0/3436550103 >> v1:172.21.15.94:6789/0 conn(0x7f26e8072260 legacy=0x7f26e815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.731+0000 7f26e6000640 1 -- 172.21.15.94:0/3436550103 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310523021 0 0) 0x7f26d4001110 con 0x7f26e8072260 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.731+0000 7f26e7e00640 1 -- 172.21.15.94:0/3436550103 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e815d950 con 0x7f26e815d2f0 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e8140a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009d90 tx=0x7f26e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9cc293c6e36167 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26e6000640 1 -- 172.21.15.94:0/3436550103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0007740 con 0x7f26e815d2f0 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26e6000640 1 -- 172.21.15.94:0/3436550103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e00078e0 con 0x7f26e815d2f0 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26e6000640 1 -- 172.21.15.94:0/3436550103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0007cb0 con 0x7f26e815d2f0 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26ed200640 1 -- 172.21.15.94:0/3436550103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 msgr2=0x7f26e8140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.732+0000 7f26ed200640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e8140a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009d90 tx=0x7f26e002a660 comp rx=0 tx=0).stop 2024-05-27T12:35:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 -- 172.21.15.94:0/3436550103 shutdown_connections 2024-05-27T12:35:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e8140a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 --2- 172.21.15.94:0/3436550103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e813f480 0x7f26e8071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 -- 172.21.15.94:0/3436550103 >> 172.21.15.94:0/3436550103 conn(0x7f26e80b6230 msgr2=0x7f26e80b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 -- 172.21.15.94:0/3436550103 shutdown_connections 2024-05-27T12:35:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.733+0000 7f26ed200640 1 -- 172.21.15.94:0/3436550103 wait complete. 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.734+0000 7f26ed200640 1 Processor -- start 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 -- start start 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e8072260 0x7f26e81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26e813f480 0x7f26e80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8140200 con 0x7f26e815d2f0 2024-05-27T12:35:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8140570 con 0x7f26e813f480 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.735+0000 7f26ed200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f26e813f860 con 0x7f26e8072260 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56860/0 (socket says 172.21.15.94:56860) 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 -- 172.21.15.94:0/3708713864 learned_addr learned my addr 172.21.15.94:0/3708713864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7400640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e8072260 0x7f26e81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e6a00640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26e813f480 0x7f26e80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 -- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e8072260 msgr2=0x7f26e81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e8072260 0x7f26e81563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 -- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26e813f480 msgr2=0x7f26e80b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26e813f480 0x7f26e80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.736+0000 7f26e7e00640 1 -- 172.21.15.94:0/3708713864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8158120 con 0x7f26e815d2f0 2024-05-27T12:35:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26e7e00640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009eb0 tx=0x7f26e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0040070 con 0x7f26e815d2f0 2024-05-27T12:35:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0031bc0 con 0x7f26e815d2f0 2024-05-27T12:35:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e8158400 con 0x7f26e815d2f0 2024-05-27T12:35:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0045460 con 0x7f26e815d2f0 2024-05-27T12:35:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.737+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81589c0 con 0x7f26e815d2f0 2024-05-27T12:35:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.738+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e8075460 con 0x7f26e815d2f0 2024-05-27T12:35:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.739+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26e0039040 con 0x7f26e815d2f0 2024-05-27T12:35:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.740+0000 7f26df400640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.740+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26e00ad4e0 con 0x7f26e815d2f0 2024-05-27T12:35:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.741+0000 7f26e7400640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.742+0000 7f26e7400640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 0x7f26c806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.744+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26e007b900 con 0x7f26e815d2f0 2024-05-27T12:35:35.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.943+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26e80ae450 con 0x7f26e815d2f0 2024-05-27T12:35:35.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.944+0000 7f26df400640 1 -- 172.21.15.94:0/3708713864 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f26e007b200 con 0x7f26e815d2f0 2024-05-27T12:35:35.943 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 msgr2=0x7f26c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 0x7f26c806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 msgr2=0x7f26e80b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009eb0 tx=0x7f26e0004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 shutdown_connections 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f26c806c8f0 0x7f26c806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26e813f480 0x7f26e80b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26e815d2f0 0x7f26e80b49f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 --2- 172.21.15.94:0/3708713864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26e8072260 0x7f26e81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:35.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.946+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 >> 172.21.15.94:0/3708713864 conn(0x7f26e80b6230 msgr2=0x7f26e80ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:35.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.947+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 shutdown_connections 2024-05-27T12:35:35.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:35.947+0000 7f26ed200640 1 -- 172.21.15.94:0/3708713864 wait complete. 2024-05-27T12:35:35.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:36.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.171+0000 7fd12fe00640 1 Processor -- start 2024-05-27T12:35:36.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.171+0000 7fd12fe00640 1 -- start start 2024-05-27T12:35:36.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 0x7fd13015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12fe00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13006dc50 con 0x7fd13015d120 2024-05-27T12:35:36.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12fe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13015d950 con 0x7fd13013f480 2024-05-27T12:35:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12fe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd13013fcd0 con 0x7fd130072430 2024-05-27T12:35:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12f400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd13015d120 0x7fd130140a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54536/0 (socket says 172.21.15.94:54536) 2024-05-27T12:35:36.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12f400640 1 -- 172.21.15.94:0/1400574089 learned_addr learned my addr 172.21.15.94:0/1400574089 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.173+0000 7fd12e000640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 0x7fd13015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858081444 0 0) 0x7fd128001490 con 0x7fd13015d120 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd114003520 con 0x7fd13015d120 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3591167447 0 0) 0x7fd114003520 con 0x7fd13015d120 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 msgr2=0x7fd13015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 0x7fd13015cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 >> v1:172.21.15.97:6789/0 conn(0x7fd13013f480 legacy=0x7fd130071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13013ff20 con 0x7fd13015d120 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fd128001bc0 con 0x7fd13015d120 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 >> v1:172.21.15.94:6789/0 conn(0x7fd13015d120 legacy=0x7fd130140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 0x7fd1140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114003f20 0x7fd114004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd114004d40 0x7fd1140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd13013fcd0 con 0x7fd114004d40 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd11400ba20 con 0x7fd114003f20 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.174+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd11400bc20 con 0x7fd130072430 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd114004d40 0x7fd1140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12e000640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114003f20 0x7fd114004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 -- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 msgr2=0x7fd1140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130072430 0x7fd1140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 -- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114003f20 msgr2=0x7fd114004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114003f20 0x7fd114004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12f400640 1 -- 172.21.15.94:0/1400574089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd114003520 con 0x7fd114004d40 2024-05-27T12:35:36.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.175+0000 7fd12e000640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114003f20 0x7fd114004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.176+0000 7fd12f400640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd114004d40 0x7fd1140051c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd12800aa50 tx=0x7fd114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838d253240d7a8ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.176+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12800a670 con 0x7fd114004d40 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.176+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12802f070 con 0x7fd114004d40 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.176+0000 7fd12d600640 1 -- 172.21.15.94:0/1400574089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128034470 con 0x7fd114004d40 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 -- 172.21.15.94:0/1400574089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd114004d40 msgr2=0x7fd1140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1400574089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd114004d40 0x7fd1140051c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd12800aa50 tx=0x7fd114005830 comp rx=0 tx=0).stop 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12f400640 1 -- 172.21.15.94:0/1400574089 reap_dead start 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 -- 172.21.15.94:0/1400574089 shutdown_connections 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 -- 172.21.15.94:0/1400574089 >> 172.21.15.94:0/1400574089 conn(0x7fd1300b6230 msgr2=0x7fd1300b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:36.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 -- 172.21.15.94:0/1400574089 shutdown_connections 2024-05-27T12:35:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.177+0000 7fd12fe00640 1 -- 172.21.15.94:0/1400574089 wait complete. 2024-05-27T12:35:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.178+0000 7fd12fe00640 1 Processor -- start 2024-05-27T12:35:36.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.179+0000 7fd12fe00640 1 -- start start 2024-05-27T12:35:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 0x7fd1301571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130131760 0x7fd130131be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd13015d570 con 0x7fd130142230 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd130140570 con 0x7fd114004d40 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12fe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd13015d6f0 con 0x7fd130131760 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56868/0 (socket says 172.21.15.94:56868) 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 -- 172.21.15.94:0/1406233266 learned_addr learned my addr 172.21.15.94:0/1406233266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12ea00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 0x7fd1301571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12f400640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130131760 0x7fd130131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 -- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130131760 msgr2=0x7fd130131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130131760 0x7fd130131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 -- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 msgr2=0x7fd1301571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 0x7fd1301571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12e000640 1 -- 172.21.15.94:0/1406233266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd130159b40 con 0x7fd130142230 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.180+0000 7fd12ea00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 0x7fd1301571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd12e000640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd11c010870 tx=0x7fd11c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00f040 con 0x7fd130142230 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c00eb60 con 0x7fd130142230 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c01c400 con 0x7fd130142230 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd130157700 con 0x7fd130142230 2024-05-27T12:35:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.181+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd130157c60 con 0x7fd130142230 2024-05-27T12:35:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.182+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd130075460 con 0x7fd130142230 2024-05-27T12:35:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.183+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd11c01c5a0 con 0x7fd130142230 2024-05-27T12:35:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.184+0000 7fd126a00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 0x7fd11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.184+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd11c08c130 con 0x7fd130142230 2024-05-27T12:35:36.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.185+0000 7fd12ea00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 0x7fd11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.186+0000 7fd12ea00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 0x7fd11006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd120001850 tx=0x7fd120009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.188+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd130157700 con 0x7fd130142230 2024-05-27T12:35:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.373+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd1300ceb00 con 0x7fd130142230 2024-05-27T12:35:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.374+0000 7fd126a00640 1 -- 172.21.15.94:0/1406233266 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd11c018030 con 0x7fd130142230 2024-05-27T12:35:36.374 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.376+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 msgr2=0x7fd11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.376+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 0x7fd11006edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd120001850 tx=0x7fd120009040 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.376+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 msgr2=0x7fd13013daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.376+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd11c010870 tx=0x7fd11c010d30 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 shutdown_connections 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd11006c8f0 0x7fd11006edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd130131760 0x7fd130131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd130142230 0x7fd13013daf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 --2- 172.21.15.94:0/1406233266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd114004d40 0x7fd1301571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 >> 172.21.15.94:0/1406233266 conn(0x7fd1300b6230 msgr2=0x7fd1301564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 shutdown_connections 2024-05-27T12:35:36.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.377+0000 7fd12fe00640 1 -- 172.21.15.94:0/1406233266 wait complete. 2024-05-27T12:35:36.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:36.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.595+0000 7fc4dea00640 1 Processor -- start 2024-05-27T12:35:36.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.595+0000 7fc4dea00640 1 -- start start 2024-05-27T12:35:36.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4d8162150 0x7fc4d80d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc4d806ca40 0x7fc4d809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dea00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d809b380 con 0x7fc4d809af70 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dea00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d80d2020 con 0x7fc4d806ca40 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dea00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d80d1d40 con 0x7fc4d8162150 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dca00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc4d809af70 0x7fc4d806d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54550/0 (socket says 172.21.15.94:54550) 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4dca00640 1 -- 172.21.15.94:0/3721127228 learned_addr learned my addr 172.21.15.94:0/3721127228 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.597+0000 7fc4d7400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc4d806ca40 0x7fc4d809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d7e00640 1 --2- 172.21.15.94:0/3721127228 >> v2:172.21.15.97:3300/0 conn(0x7fc4d8162150 0x7fc4d80d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d7400640 1 -- 172.21.15.94:0/3721127228 >> v2:172.21.15.97:3300/0 conn(0x7fc4d8162150 msgr2=0x7fc4d80d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d7400640 1 --2- 172.21.15.94:0/3721127228 >> v2:172.21.15.97:3300/0 conn(0x7fc4d8162150 0x7fc4d80d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d7400640 1 -- 172.21.15.94:0/3721127228 >> v1:172.21.15.94:6789/0 conn(0x7fc4d809af70 legacy=0x7fc4d806d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d7400640 1 -- 172.21.15.94:0/3721127228 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d80ccd60 con 0x7fc4d806ca40 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.598+0000 7fc4d6a00640 1 -- 172.21.15.94:0/3721127228 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103413171 0 0) 0x7fc4d0001490 con 0x7fc4d809af70 2024-05-27T12:35:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.599+0000 7fc4d7400640 1 --2- 172.21.15.94:0/3721127228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d806ca40 0x7fc4d809aa30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b4130 tx=0x7fc4c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1152565a240029a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.599+0000 7fc4d6a00640 1 -- 172.21.15.94:0/3721127228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0037070 con 0x7fc4d806ca40 2024-05-27T12:35:36.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.599+0000 7fc4d6a00640 1 -- 172.21.15.94:0/3721127228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c0032440 con 0x7fc4d806ca40 2024-05-27T12:35:36.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.599+0000 7fc4d6a00640 1 -- 172.21.15.94:0/3721127228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0032810 con 0x7fc4d806ca40 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 -- 172.21.15.94:0/3721127228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d806ca40 msgr2=0x7fc4d809aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 --2- 172.21.15.94:0/3721127228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d806ca40 0x7fc4d809aa30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b4130 tx=0x7fc4c002a1a0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 -- 172.21.15.94:0/3721127228 shutdown_connections 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 --2- 172.21.15.94:0/3721127228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d806ca40 0x7fc4d809aa30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 --2- 172.21.15.94:0/3721127228 >> v2:172.21.15.97:3300/0 conn(0x7fc4d8162150 0x7fc4d80d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 -- 172.21.15.94:0/3721127228 >> 172.21.15.94:0/3721127228 conn(0x7fc4d80b4160 msgr2=0x7fc4d80b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 -- 172.21.15.94:0/3721127228 shutdown_connections 2024-05-27T12:35:36.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.600+0000 7fc4dea00640 1 -- 172.21.15.94:0/3721127228 wait complete. 2024-05-27T12:35:36.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.602+0000 7fc4dea00640 1 Processor -- start 2024-05-27T12:35:36.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.602+0000 7fc4dea00640 1 -- start start 2024-05-27T12:35:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4d806ca40 0x7fc4d815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4d8162150 0x7fc4d80b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d80d1a30 con 0x7fc4d809af70 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d80cd320 con 0x7fc4d8162150 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4dea00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d80ce850 con 0x7fc4d806ca40 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56882/0 (socket says 172.21.15.94:56882) 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 -- 172.21.15.94:0/1519497233 learned_addr learned my addr 172.21.15.94:0/1519497233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 -- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4d806ca40 msgr2=0x7fc4d815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4d806ca40 0x7fc4d815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 -- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4d8162150 msgr2=0x7fc4d80b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4d8162150 0x7fc4d80b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.603+0000 7fc4d7400640 1 -- 172.21.15.94:0/1519497233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d80c8010 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4d7400640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009890 tx=0x7fc4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0037040 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c0042070 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d80cba00 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0030bd0 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.604+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d80c6f70 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.605+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d8068a70 con 0x7fc4d809af70 2024-05-27T12:35:36.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.606+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4c0047460 con 0x7fc4d809af70 2024-05-27T12:35:36.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.607+0000 7fc4cfe00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:36.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.607+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc4c00ac7c0 con 0x7fc4d809af70 2024-05-27T12:35:36.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.608+0000 7fc4d7e00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:36.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.609+0000 7fc4d7e00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006fd0 tx=0x7fc4c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:36.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.611+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4c0032e20 con 0x7fc4d809af70 2024-05-27T12:35:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.796+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4d80d22a0 con 0x7fc4d809af70 2024-05-27T12:35:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.796+0000 7fc4cfe00640 1 -- 172.21.15.94:0/1519497233 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc4c007ab50 con 0x7fc4d809af70 2024-05-27T12:35:36.796 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.798+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 msgr2=0x7fc4b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.798+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006fd0 tx=0x7fc4c8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.798+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 msgr2=0x7fc4d80cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.798+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009890 tx=0x7fc4c00046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 shutdown_connections 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4d8162150 0x7fc4d80b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d809af70 0x7fc4d80cb4c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 --2- 172.21.15.94:0/1519497233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4d806ca40 0x7fc4d815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 >> 172.21.15.94:0/1519497233 conn(0x7fc4d80b4160 msgr2=0x7fc4d809b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 shutdown_connections 2024-05-27T12:35:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:36.799+0000 7fc4dea00640 1 -- 172.21.15.94:0/1519497233 wait complete. 2024-05-27T12:35:36.893 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that when one clone is in progress, one progress bar is printed 2024-05-27T12:35:36.893 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-27T12:35:36.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:37.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.100+0000 7fceb4c00640 1 Processor -- start 2024-05-27T12:35:37.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.100+0000 7fceb4c00640 1 -- start start 2024-05-27T12:35:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fceb4c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fceb0164360 0x7fceb0166750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fceb4c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb00b6b80 con 0x7fceb00d0650 2024-05-27T12:35:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fceb4c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb00cf1a0 con 0x7fceb0145140 2024-05-27T12:35:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fceb4c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fceb0145520 con 0x7fceb0164360 2024-05-27T12:35:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fcea5400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fceb00d0650 0x7fceb00d0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54566/0 (socket says 172.21.15.94:54566) 2024-05-27T12:35:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.102+0000 7fcea5400640 1 -- 172.21.15.94:0/512303276 learned_addr learned my addr 172.21.15.94:0/512303276 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fceae800640 1 --2- 172.21.15.94:0/512303276 >> v2:172.21.15.97:3300/0 conn(0x7fceb0164360 0x7fceb0166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 921620531 0 0) 0x7fceb00b6b80 con 0x7fceb00d0650 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce94003520 con 0x7fceb00d0650 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4054441918 0 0) 0x7fce94003520 con 0x7fceb00d0650 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 >> v2:172.21.15.97:3300/0 conn(0x7fceb0164360 msgr2=0x7fceb0166750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 --2- 172.21.15.94:0/512303276 >> v2:172.21.15.97:3300/0 conn(0x7fceb0164360 0x7fceb0166750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 >> v1:172.21.15.97:6789/0 conn(0x7fceb0145140 legacy=0x7fceb00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb00b5ae0 con 0x7fceb00d0650 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fce98001690 con 0x7fceb00d0650 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 >> v1:172.21.15.94:6789/0 conn(0x7fceb00d0650 legacy=0x7fceb00d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce94003ae0 0x7fce94003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 0x7fce94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce940052e0 0x7fce9400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0145520 con 0x7fce94004800 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce9400e440 con 0x7fce940052e0 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.103+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fce9400e640 con 0x7fce94003ae0 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 0x7fce94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fceade00640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce94003ae0 0x7fce94003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 -- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce94003ae0 msgr2=0x7fce94003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fceae800640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce940052e0 0x7fce9400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce94003ae0 0x7fce94003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 -- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce940052e0 msgr2=0x7fce9400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:37.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce940052e0 0x7fce9400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fceae800640 1 -- 172.21.15.94:0/512303276 reap_dead start 2024-05-27T12:35:37.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fcea5400640 1 -- 172.21.15.94:0/512303276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce94003520 con 0x7fce94004800 2024-05-27T12:35:37.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.104+0000 7fceade00640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce94003ae0 0x7fce94003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.105+0000 7fcea5400640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 0x7fce94004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b6d50 tx=0x7fce94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ea8009cd8b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.105+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98007c60 con 0x7fce94004800 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.105+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce98005ad0 con 0x7fce94004800 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.105+0000 7fcead400640 1 -- 172.21.15.94:0/512303276 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98007070 con 0x7fce94004800 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 -- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 msgr2=0x7fce94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 0x7fce94004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b6d50 tx=0x7fce94005830 comp rx=0 tx=0).stop 2024-05-27T12:35:37.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 -- 172.21.15.94:0/512303276 shutdown_connections 2024-05-27T12:35:37.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 --2- 172.21.15.94:0/512303276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce94004800 0x7fce94004c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 -- 172.21.15.94:0/512303276 >> 172.21.15.94:0/512303276 conn(0x7fceb00b3360 msgr2=0x7fceb00b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:37.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 -- 172.21.15.94:0/512303276 shutdown_connections 2024-05-27T12:35:37.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.106+0000 7fceb4c00640 1 -- 172.21.15.94:0/512303276 wait complete. 2024-05-27T12:35:37.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.108+0000 7fceb4c00640 1 Processor -- start 2024-05-27T12:35:37.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.109+0000 7fceb4c00640 1 -- start start 2024-05-27T12:35:37.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 0x7fceb0140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fceb0142c20 0x7fceb01430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00b6380 con 0x7fceb00b1ab0 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00b60a0 con 0x7fce94004800 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceb4c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00b6660 con 0x7fceb0142c20 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56894/0 (socket says 172.21.15.94:56894) 2024-05-27T12:35:37.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 -- 172.21.15.94:0/216212616 learned_addr learned my addr 172.21.15.94:0/216212616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceade00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 0x7fceb0140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 -- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fceb0142c20 msgr2=0x7fceb01430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fceb0142c20 0x7fceb01430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 -- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 msgr2=0x7fceb0140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 0x7fceb0140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fcea5400640 1 -- 172.21.15.94:0/216212616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb01587b0 con 0x7fceb00b1ab0 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.110+0000 7fceade00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 0x7fceb0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fcea5400640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fceb00b5cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9803f020 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce98001490 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01435e0 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98007070 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.111+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb0141890 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.112+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0141a90 con 0x7fceb00b1ab0 2024-05-27T12:35:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.113+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce98042020 con 0x7fceb00b1ab0 2024-05-27T12:35:37.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.115+0000 7fcea6a00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.115+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce980ab830 con 0x7fceb00b1ab0 2024-05-27T12:35:37.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.115+0000 7fceade00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.116+0000 7fceade00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 0x7fce8c06ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b23c0 tx=0x7fcea0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.119+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fceb0141a90 con 0x7fceb00b1ab0 2024-05-27T12:35:37.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.315+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fceb0075460 con 0x7fceb00b1ab0 2024-05-27T12:35:37.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.318+0000 7fcea6a00640 1 -- 172.21.15.94:0/216212616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fceb01587b0 con 0x7fceb00b1ab0 2024-05-27T12:35:37.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:37.318 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:37.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 msgr2=0x7fce8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 0x7fce8c06ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b23c0 tx=0x7fcea0009040 comp rx=0 tx=0).stop 2024-05-27T12:35:37.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 msgr2=0x7fceb0158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fceb00b5cb0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 shutdown_connections 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fceb0142c20 0x7fceb01430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb00b1ab0 0x7fceb0158270 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 --2- 172.21.15.94:0/216212616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94004800 0x7fceb0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 >> 172.21.15.94:0/216212616 conn(0x7fceb00b3360 msgr2=0x7fceb0163bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 shutdown_connections 2024-05-27T12:35:37.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.321+0000 7fceb4c00640 1 -- 172.21.15.94:0/216212616 wait complete. 2024-05-27T12:35:37.338 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:37.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:37.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.542+0000 7fd76f200640 1 Processor -- start 2024-05-27T12:35:37.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.543+0000 7fd76f200640 1 -- start start 2024-05-27T12:35:37.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76f200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd7700ba3f0 0x7fd7700b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7700b66d0 0x7fd7700d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76f200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7700b7840 con 0x7fd77015e900 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76f200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd7700b6b00 con 0x7fd7700ba3f0 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76f200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fd770067f30 con 0x7fd7700b66d0 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76de00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd77015e900 0x7fd770067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54574/0 (socket says 172.21.15.94:54574) 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76de00640 1 -- 172.21.15.94:0/3447127519 learned_addr learned my addr 172.21.15.94:0/3447127519 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:37.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.544+0000 7fd76d400640 1 --2- 172.21.15.94:0/3447127519 >> v2:172.21.15.94:3300/0 conn(0x7fd7700ba3f0 0x7fd7700b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76e800640 1 --2- 172.21.15.94:0/3447127519 >> v2:172.21.15.97:3300/0 conn(0x7fd7700b66d0 0x7fd7700d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76ca00640 1 -- 172.21.15.94:0/3447127519 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249159030 0 0) 0x7fd758001110 con 0x7fd77015e900 2024-05-27T12:35:37.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76ca00640 1 -- 172.21.15.94:0/3447127519 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd754003520 con 0x7fd77015e900 2024-05-27T12:35:37.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76d400640 1 -- 172.21.15.94:0/3447127519 >> v2:172.21.15.97:3300/0 conn(0x7fd7700b66d0 msgr2=0x7fd7700d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:37.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76d400640 1 --2- 172.21.15.94:0/3447127519 >> v2:172.21.15.97:3300/0 conn(0x7fd7700b66d0 0x7fd7700d3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76d400640 1 -- 172.21.15.94:0/3447127519 >> v1:172.21.15.94:6789/0 conn(0x7fd77015e900 legacy=0x7fd770067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.545+0000 7fd76d400640 1 -- 172.21.15.94:0/3447127519 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd770068210 con 0x7fd7700ba3f0 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.546+0000 7fd76d400640 1 --2- 172.21.15.94:0/3447127519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd7700b6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7700b72e0 tx=0x7fd76002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bf02e23fb4144 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.546+0000 7fd76ca00640 1 -- 172.21.15.94:0/3447127519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760037070 con 0x7fd7700ba3f0 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.546+0000 7fd76ca00640 1 -- 172.21.15.94:0/3447127519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd760032440 con 0x7fd7700ba3f0 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.546+0000 7fd76ca00640 1 -- 172.21.15.94:0/3447127519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760032810 con 0x7fd7700ba3f0 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 -- 172.21.15.94:0/3447127519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 msgr2=0x7fd7700b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 --2- 172.21.15.94:0/3447127519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd7700b6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7700b72e0 tx=0x7fd76002a230 comp rx=0 tx=0).stop 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 -- 172.21.15.94:0/3447127519 shutdown_connections 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 --2- 172.21.15.94:0/3447127519 >> v2:172.21.15.97:3300/0 conn(0x7fd7700b66d0 0x7fd7700d3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 --2- 172.21.15.94:0/3447127519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd7700b6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 -- 172.21.15.94:0/3447127519 >> 172.21.15.94:0/3447127519 conn(0x7fd7700d1190 msgr2=0x7fd7700d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 -- 172.21.15.94:0/3447127519 shutdown_connections 2024-05-27T12:35:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.547+0000 7fd76f200640 1 -- 172.21.15.94:0/3447127519 wait complete. 2024-05-27T12:35:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.548+0000 7fd76f200640 1 Processor -- start 2024-05-27T12:35:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 -- start start 2024-05-27T12:35:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7700b66d0 0x7fd770157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77015e900 0x7fd7701566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700d2df0 con 0x7fd7700ba3f0 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7700d2b10 con 0x7fd77015e900 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.549+0000 7fd76f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd77015ece0 con 0x7fd7700b66d0 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56924/0 (socket says 172.21.15.94:56924) 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 -- 172.21.15.94:0/3102373545 learned_addr learned my addr 172.21.15.94:0/3102373545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 -- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7700b66d0 msgr2=0x7fd770157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7700b66d0 0x7fd770157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 -- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77015e900 msgr2=0x7fd7701566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77015e900 0x7fd7701566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 -- 172.21.15.94:0/3102373545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd770158310 con 0x7fd7700ba3f0 2024-05-27T12:35:37.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.550+0000 7fd76d400640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd760000c00 tx=0x7fd7600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.551+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760037040 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.551+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd760042070 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.551+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd770161fc0 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.551+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760030960 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.551+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd770157000 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.552+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770075460 con 0x7fd7700ba3f0 2024-05-27T12:35:37.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.553+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd760030b00 con 0x7fd7700ba3f0 2024-05-27T12:35:37.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.554+0000 7fd766000640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 0x7fd74c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.554+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd7600ac470 con 0x7fd7700ba3f0 2024-05-27T12:35:37.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.554+0000 7fd76de00640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 0x7fd74c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.556+0000 7fd76de00640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 0x7fd74c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd758000f60 tx=0x7fd758001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.558+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd76007a890 con 0x7fd7700ba3f0 2024-05-27T12:35:37.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.753+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7700d0020 con 0x7fd7700ba3f0 2024-05-27T12:35:37.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.754+0000 7fd766000640 1 -- 172.21.15.94:0/3102373545 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd76007a190 con 0x7fd7700ba3f0 2024-05-27T12:35:37.753 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 msgr2=0x7fd74c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 0x7fd74c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd758000f60 tx=0x7fd758001110 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 msgr2=0x7fd770156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd760000c00 tx=0x7fd7600046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 shutdown_connections 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd74c06c8f0 0x7fd74c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77015e900 0x7fd7701566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7700ba3f0 0x7fd770156180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 --2- 172.21.15.94:0/3102373545 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7700b66d0 0x7fd770157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 >> 172.21.15.94:0/3102373545 conn(0x7fd7700d1190 msgr2=0x7fd7700d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 shutdown_connections 2024-05-27T12:35:37.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.756+0000 7fd76f200640 1 -- 172.21.15.94:0/3102373545 wait complete. 2024-05-27T12:35:37.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:37.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.977+0000 7f22aa800640 1 Processor -- start 2024-05-27T12:35:37.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.978+0000 7f22aa800640 1 -- start start 2024-05-27T12:35:37.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.979+0000 7f22aa800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.979+0000 7f22aa800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22a40ce750 0x7f22a4071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.979+0000 7f22aa800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a406e4a0 con 0x7f22a4162ad0 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.979+0000 7f22aa800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f22a40b38b0 con 0x7f22a40ce750 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.979+0000 7f22aa800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f22a406e780 con 0x7f22a4162eb0 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.980+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.980+0000 7f22a9e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22a40ce750 0x7f22a4071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.980+0000 7f22a8a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:41774/0 (socket says 172.21.15.94:41774) 2024-05-27T12:35:37.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.980+0000 7f22a8a00640 1 -- 172.21.15.94:0/3279841796 learned_addr learned my addr 172.21.15.94:0/3279841796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:37.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.980+0000 7f22a9e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22a40ce750 0x7f22a4071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:51988/0 (socket says 172.21.15.94:51988) 2024-05-27T12:35:37.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a9e00640 1 -- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 msgr2=0x7f22a40ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:37.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a9e00640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a9e00640 1 -- 172.21.15.94:0/3279841796 >> v1:172.21.15.97:6789/0 conn(0x7f22a4162ad0 legacy=0x7f22a406dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a9e00640 1 -- 172.21.15.94:0/3279841796 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a406ea60 con 0x7f22a40ce750 2024-05-27T12:35:37.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a8a00640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:35:37.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.981+0000 7f22a3e00640 1 -- 172.21.15.94:0/3279841796 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375800568 0 0) 0x7f22a406e4a0 con 0x7f22a4162ad0 2024-05-27T12:35:37.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.982+0000 7f22a9e00640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a40ce750 0x7f22a4071d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f229c009d90 tx=0x7f229c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf3b6a2d2e84cb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.983+0000 7f22a3e00640 1 -- 172.21.15.94:0/3279841796 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007740 con 0x7f22a40ce750 2024-05-27T12:35:37.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.983+0000 7f22a3e00640 1 -- 172.21.15.94:0/3279841796 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229c0078e0 con 0x7f22a40ce750 2024-05-27T12:35:37.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.983+0000 7f22a3e00640 1 -- 172.21.15.94:0/3279841796 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007cb0 con 0x7f22a40ce750 2024-05-27T12:35:37.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 -- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a40ce750 msgr2=0x7f22a4071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a40ce750 0x7f22a4071d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f229c009d90 tx=0x7f229c02a660 comp rx=0 tx=0).stop 2024-05-27T12:35:37.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 -- 172.21.15.94:0/3279841796 shutdown_connections 2024-05-27T12:35:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a40ce750 0x7f22a4071d20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 --2- 172.21.15.94:0/3279841796 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162eb0 0x7f22a40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 -- 172.21.15.94:0/3279841796 >> 172.21.15.94:0/3279841796 conn(0x7f22a40b6230 msgr2=0x7f22a40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 -- 172.21.15.94:0/3279841796 shutdown_connections 2024-05-27T12:35:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.984+0000 7f22aa800640 1 -- 172.21.15.94:0/3279841796 wait complete. 2024-05-27T12:35:37.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.986+0000 7f22aa800640 1 Processor -- start 2024-05-27T12:35:37.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.986+0000 7f22aa800640 1 -- start start 2024-05-27T12:35:37.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162ad0 0x7f22a415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 0x7f22a41fc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22a406f410 con 0x7f22a40ce750 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22a406f290 con 0x7f22a41fbb60 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22aa800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f22a406f590 con 0x7f22a4162ad0 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56934/0 (socket says 172.21.15.94:56934) 2024-05-27T12:35:37.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.987+0000 7f22a9400640 1 -- 172.21.15.94:0/2307507324 learned_addr learned my addr 172.21.15.94:0/2307507324 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:37.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9e00640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 0x7f22a41fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a8a00640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162ad0 0x7f22a415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 -- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162ad0 msgr2=0x7f22a415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162ad0 0x7f22a415db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 -- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 msgr2=0x7f22a41fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 0x7f22a41fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 -- 172.21.15.94:0/2307507324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41fc620 con 0x7f22a40ce750 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9e00640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 0x7f22a41fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a9400640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f229800c4b0 tx=0x7f229800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298017070 con 0x7f22a40ce750 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298005480 con 0x7f22a40ce750 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298003830 con 0x7f22a40ce750 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.988+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41fc900 con 0x7f22a40ce750 2024-05-27T12:35:37.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.989+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a40ad230 con 0x7f22a40ce750 2024-05-27T12:35:37.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.989+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a40ad430 con 0x7f22a40ce750 2024-05-27T12:35:37.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.991+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f229801a070 con 0x7f22a40ce750 2024-05-27T12:35:37.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.991+0000 7f22a1600640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 0x7f228406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:37.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.992+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f229808af90 con 0x7f22a40ce750 2024-05-27T12:35:37.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.992+0000 7f22a8a00640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 0x7f228406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:37.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.993+0000 7f22a8a00640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 0x7f228406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b0720 tx=0x7f2290008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:37.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:37.996+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22a40ad430 con 0x7f22a40ce750 2024-05-27T12:35:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.196+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22a4075460 con 0x7f22a40ce750 2024-05-27T12:35:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.197+0000 7f22a1600640 1 -- 172.21.15.94:0/2307507324 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2298059320 con 0x7f22a40ce750 2024-05-27T12:35:38.197 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.199+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 msgr2=0x7f228406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.199+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 0x7f228406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f22a40b0720 tx=0x7f2290008040 comp rx=0 tx=0).stop 2024-05-27T12:35:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.199+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 msgr2=0x7f22a40cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.199+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f229800c4b0 tx=0x7f229800c970 comp rx=0 tx=0).stop 2024-05-27T12:35:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 shutdown_connections 2024-05-27T12:35:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f228406c8f0 0x7f228406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22a41fbb60 0x7f22a41fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22a4162ad0 0x7f22a415db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 --2- 172.21.15.94:0/2307507324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a40ce750 0x7f22a40cc530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 >> 172.21.15.94:0/2307507324 conn(0x7f22a40b6230 msgr2=0x7f22a406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 shutdown_connections 2024-05-27T12:35:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.200+0000 7f22aa800640 1 -- 172.21.15.94:0/2307507324 wait complete. 2024-05-27T12:35:38.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:38.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.414+0000 7f0afe400640 1 Processor -- start 2024-05-27T12:35:38.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.414+0000 7f0afe400640 1 -- start start 2024-05-27T12:35:38.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0af8146060 0x7f0af80b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0af80cd8a0 0x7f0af806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af80cdd30 0x7f0af806be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f0af815dbe0 con 0x7f0af80cd8a0 2024-05-27T12:35:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0af815cfd0 con 0x7f0af8146060 2024-05-27T12:35:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0afe400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0af815d2b0 con 0x7f0af80cdd30 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0aefe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0af80cd8a0 0x7f0af806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0aefe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f0af80cd8a0 0x7f0af806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56936/0 (socket says 172.21.15.94:56936) 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.416+0000 7f0aefe00640 1 -- 172.21.15.94:0/3733934316 learned_addr learned my addr 172.21.15.94:0/3733934316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0af7400640 1 --2- 172.21.15.94:0/3733934316 >> v2:172.21.15.97:3300/0 conn(0x7f0af8146060 0x7f0af80b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0af7e00640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af80cdd30 0x7f0af806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0aefe00640 1 -- 172.21.15.94:0/3733934316 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af80cdd30 msgr2=0x7f0af806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0aefe00640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af80cdd30 0x7f0af806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0aefe00640 1 -- 172.21.15.94:0/3733934316 >> v2:172.21.15.97:3300/0 conn(0x7f0af8146060 msgr2=0x7f0af80b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0aefe00640 1 --2- 172.21.15.94:0/3733934316 >> v2:172.21.15.97:3300/0 conn(0x7f0af8146060 0x7f0af80b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.417+0000 7f0aefe00640 1 -- 172.21.15.94:0/3733934316 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af815d590 con 0x7f0af80cd8a0 2024-05-27T12:35:38.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.418+0000 7f0aefe00640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cd8a0 0x7f0af806b930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0af80b5c30 tx=0x7f0ae002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc255b5ec0c4c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.418+0000 7f0af6a00640 1 -- 172.21.15.94:0/3733934316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0037070 con 0x7f0af80cd8a0 2024-05-27T12:35:38.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0af6a00640 1 -- 172.21.15.94:0/3733934316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae0032440 con 0x7f0af80cd8a0 2024-05-27T12:35:38.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0af6a00640 1 -- 172.21.15.94:0/3733934316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0032810 con 0x7f0af80cd8a0 2024-05-27T12:35:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 -- 172.21.15.94:0/3733934316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cd8a0 msgr2=0x7f0af806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cd8a0 0x7f0af806b930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0af80b5c30 tx=0x7f0ae002a140 comp rx=0 tx=0).stop 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 -- 172.21.15.94:0/3733934316 shutdown_connections 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af80cdd30 0x7f0af806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 --2- 172.21.15.94:0/3733934316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cd8a0 0x7f0af806b930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 --2- 172.21.15.94:0/3733934316 >> v2:172.21.15.97:3300/0 conn(0x7f0af8146060 0x7f0af80b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 -- 172.21.15.94:0/3733934316 >> 172.21.15.94:0/3733934316 conn(0x7f0af80b2240 msgr2=0x7f0af80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.419+0000 7f0afe400640 1 -- 172.21.15.94:0/3733934316 shutdown_connections 2024-05-27T12:35:38.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.420+0000 7f0afe400640 1 -- 172.21.15.94:0/3733934316 wait complete. 2024-05-27T12:35:38.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.421+0000 7f0afe400640 1 Processor -- start 2024-05-27T12:35:38.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.421+0000 7f0afe400640 1 -- start start 2024-05-27T12:35:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af80cd8a0 0x7f0af81416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af8146060 0x7f0af813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0af80d0110 con 0x7f0af80cdd30 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0af8146440 con 0x7f0af80cd8a0 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.422+0000 7f0afe400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0af8072010 con 0x7f0af8146060 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56944/0 (socket says 172.21.15.94:56944) 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 -- 172.21.15.94:0/318435787 learned_addr learned my addr 172.21.15.94:0/318435787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0af7400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af80cd8a0 0x7f0af81416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 -- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af8146060 msgr2=0x7f0af813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af8146060 0x7f0af813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 -- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af80cd8a0 msgr2=0x7f0af81416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af80cd8a0 0x7f0af81416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 -- 172.21.15.94:0/318435787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af806f530 con 0x7f0af80cdd30 2024-05-27T12:35:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aefe00640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002a180 tx=0x7f0ae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.423+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0041020 con 0x7f0af80cdd30 2024-05-27T12:35:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.424+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae00306c0 con 0x7f0af80cdd30 2024-05-27T12:35:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.424+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0037040 con 0x7f0af80cdd30 2024-05-27T12:35:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.424+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af80ade30 con 0x7f0af80cdd30 2024-05-27T12:35:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.424+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af80ae3f0 con 0x7f0af80cdd30 2024-05-27T12:35:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.425+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8068a70 con 0x7f0af80cdd30 2024-05-27T12:35:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.426+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ae0044020 con 0x7f0af80cdd30 2024-05-27T12:35:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.427+0000 7f0aef400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.427+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ae00ac3b0 con 0x7f0af80cdd30 2024-05-27T12:35:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.427+0000 7f0af7400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.428+0000 7f0af7400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8006fd0 tx=0x7f0ae8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.431+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ae004b030 con 0x7f0af80cdd30 2024-05-27T12:35:38.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.612+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0af80b3840 con 0x7f0af80cdd30 2024-05-27T12:35:38.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.613+0000 7f0aef400640 1 -- 172.21.15.94:0/318435787 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0ae007a7d0 con 0x7f0af80cdd30 2024-05-27T12:35:38.612 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 msgr2=0x7f0ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8006fd0 tx=0x7f0ae8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 msgr2=0x7f0af813e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002a180 tx=0x7f0ae00046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 shutdown_connections 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0af8146060 0x7f0af813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af80cdd30 0x7f0af813e050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 --2- 172.21.15.94:0/318435787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af80cd8a0 0x7f0af81416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.615+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 >> 172.21.15.94:0/318435787 conn(0x7f0af80b2240 msgr2=0x7f0af80ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:38.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.616+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 shutdown_connections 2024-05-27T12:35:38.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.616+0000 7f0afe400640 1 -- 172.21.15.94:0/318435787 wait complete. 2024-05-27T12:35:38.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-27T12:35:38.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.829+0000 7f2f9de00640 1 Processor -- start 2024-05-27T12:35:38.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.829+0000 7f2f9de00640 1 -- start start 2024-05-27T12:35:38.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9de00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f980cdb90 0x7f2f98071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9de00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9813e9e0 con 0x7f2f9813e600 2024-05-27T12:35:38.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9de00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9815ba50 con 0x7f2f9806f5e0 2024-05-27T12:35:38.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9de00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2f9815bd10 con 0x7f2f980cdb90 2024-05-27T12:35:38.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9ca00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2f9813e600 0x7f2f9806eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54590/0 (socket says 172.21.15.94:54590) 2024-05-27T12:35:38.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9ca00640 1 -- 172.21.15.94:0/2130830346 learned_addr learned my addr 172.21.15.94:0/2130830346 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:38.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f9d400640 1 --2- 172.21.15.94:0/2130830346 >> v2:172.21.15.97:3300/0 conn(0x7f2f980cdb90 0x7f2f98071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.831+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288376945 0 0) 0x7f2f88001110 con 0x7f2f9813e600 2024-05-27T12:35:38.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f80007930 con 0x7f2f9813e600 2024-05-27T12:35:38.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2148235853 0 0) 0x7f2f80007930 con 0x7f2f9813e600 2024-05-27T12:35:38.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 >> v2:172.21.15.97:3300/0 conn(0x7f2f980cdb90 msgr2=0x7f2f98071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 --2- 172.21.15.94:0/2130830346 >> v2:172.21.15.97:3300/0 conn(0x7f2f980cdb90 0x7f2f98071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 >> v1:172.21.15.97:6789/0 conn(0x7f2f9806f5e0 legacy=0x7f2f980cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9815bff0 con 0x7f2f9813e600 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f2f88001840 con 0x7f2f9813e600 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 >> v1:172.21.15.94:6789/0 conn(0x7f2f9813e600 legacy=0x7f2f9806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80007dc0 0x7f2f800081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 0x7f2f80009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f800095d0 0x7f2f8000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9815bd10 con 0x7f2f80008c10 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80010470 con 0x7f2f800095d0 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f80010670 con 0x7f2f80007dc0 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.832+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 0x7f2f80009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f97e00640 1 -- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80007dc0 msgr2=0x7f2f800081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80007dc0 0x7f2f800081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f9d400640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f800095d0 0x7f2f8000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f97e00640 1 -- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f800095d0 msgr2=0x7f2f8000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f800095d0 0x7f2f8000fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f97e00640 1 -- 172.21.15.94:0/2130830346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f80007930 con 0x7f2f80008c10 2024-05-27T12:35:38.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.833+0000 7f2f9d400640 1 -- 172.21.15.94:0/2130830346 reap_dead start 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.834+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 0x7f2f80009090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f9806fdf0 tx=0x7f2f80009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907b79c97305b07e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.834+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c001370 con 0x7f2f80008c10 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.834+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0052d0 con 0x7f2f80008c10 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.834+0000 7f2f97400640 1 -- 172.21.15.94:0/2130830346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c02ab70 con 0x7f2f80008c10 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 -- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 msgr2=0x7f2f80009090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 0x7f2f80009090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f9806fdf0 tx=0x7f2f80009c40 comp rx=0 tx=0).stop 2024-05-27T12:35:38.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 -- 172.21.15.94:0/2130830346 shutdown_connections 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2130830346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f80008c10 0x7f2f80009090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 -- 172.21.15.94:0/2130830346 >> 172.21.15.94:0/2130830346 conn(0x7f2f980b2160 msgr2=0x7f2f980b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 -- 172.21.15.94:0/2130830346 shutdown_connections 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.835+0000 7f2f9de00640 1 -- 172.21.15.94:0/2130830346 wait complete. 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.837+0000 7f2f9de00640 1 Processor -- start 2024-05-27T12:35:38.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.837+0000 7f2f9de00640 1 -- start start 2024-05-27T12:35:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 0x7f2f980ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f9814bd50 0x7f2f9806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9815c920 con 0x7f2f9814a510 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9815c5b0 con 0x7f2f9814bd50 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.838+0000 7f2f9de00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f98071d60 con 0x7f2f80008c10 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56962/0 (socket says 172.21.15.94:56962) 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 -- 172.21.15.94:0/2953443977 learned_addr learned my addr 172.21.15.94:0/2953443977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f9ca00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 0x7f2f980ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 -- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 msgr2=0x7f2f980ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 0x7f2f980ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f9d400640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f9814bd50 0x7f2f9806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 -- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f9814bd50 msgr2=0x7f2f9806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f9814bd50 0x7f2f9806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 -- 172.21.15.94:0/2953443977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9813e1b0 con 0x7f2f9814a510 2024-05-27T12:35:38.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f9ca00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 0x7f2f980ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.839+0000 7f2f97e00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c001810 tx=0x7f2f8c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.840+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c03f030 con 0x7f2f9814a510 2024-05-27T12:35:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.840+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c005480 con 0x7f2f9814a510 2024-05-27T12:35:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.840+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c037040 con 0x7f2f9814a510 2024-05-27T12:35:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.840+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9814acb0 con 0x7f2f9814a510 2024-05-27T12:35:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.840+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9814b270 con 0x7f2f9814a510 2024-05-27T12:35:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.841+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98075460 con 0x7f2f9814a510 2024-05-27T12:35:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.842+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f8c042070 con 0x7f2f9814a510 2024-05-27T12:35:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.843+0000 7f2f94c00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.843+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f8c0ab5e0 con 0x7f2f9814a510 2024-05-27T12:35:38.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.844+0000 7f2f9ca00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:38.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.845+0000 7f2f9ca00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001eb0 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:38.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:38.847+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f9813e1b0 con 0x7f2f9814a510 2024-05-27T12:35:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.122+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f2f9806edd0 con 0x7f2f9814a510 2024-05-27T12:35:39.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.155+0000 7f2f94c00640 1 -- 172.21.15.94:0/2953443977 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v54) ==== 163+0+0 (secure 0 0 0) 0x7f2f8c0433b0 con 0x7f2f9814a510 2024-05-27T12:35:39.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.157+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 msgr2=0x7f2f7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.157+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001eb0 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2024-05-27T12:35:39.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.157+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 msgr2=0x7f2f9813dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.157+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c001810 tx=0x7f2f8c001120 comp rx=0 tx=0).stop 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 shutdown_connections 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f9814bd50 0x7f2f9806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9814a510 0x7f2f9813dc70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 --2- 172.21.15.94:0/2953443977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f80008c10 0x7f2f980ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 >> 172.21.15.94:0/2953443977 conn(0x7f2f980b2160 msgr2=0x7f2f980b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 shutdown_connections 2024-05-27T12:35:39.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.158+0000 7f2f9de00640 1 -- 172.21.15.94:0/2953443977 wait complete. 2024-05-27T12:35:39.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-27T12:35:39.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.372+0000 7fe7b8a00640 1 Processor -- start 2024-05-27T12:35:39.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.373+0000 7fe7b8a00640 1 -- start start 2024-05-27T12:35:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b8a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b40ba3f0 0x7fe7b40b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b8a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b40b7840 con 0x7fe7b415e900 2024-05-27T12:35:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b8a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b4067f30 con 0x7fe7b40b66d0 2024-05-27T12:35:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b8a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b4068210 con 0x7fe7b40ba3f0 2024-05-27T12:35:39.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b2800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fe7b415e900 0x7fe7b4067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54596/0 (socket says 172.21.15.94:54596) 2024-05-27T12:35:39.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.374+0000 7fe7b2800640 1 -- 172.21.15.94:0/1994397594 learned_addr learned my addr 172.21.15.94:0/1994397594 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437393684 0 0) 0x7fe7a4001110 con 0x7fe7b415e900 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe794003520 con 0x7fe7b415e900 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b40ba3f0 0x7fe7b40b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1399679271 0 0) 0x7fe794003520 con 0x7fe7b415e900 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b40ba3f0 msgr2=0x7fe7b40b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b40ba3f0 0x7fe7b40b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 >> v1:172.21.15.97:6789/0 conn(0x7fe7b40b66d0 legacy=0x7fe7b40d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b40684f0 con 0x7fe7b415e900 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fe7a4001840 con 0x7fe7b415e900 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 >> v1:172.21.15.94:6789/0 conn(0x7fe7b415e900 legacy=0x7fe7b4067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7940039e0 0x7fe794003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe794004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7940051c0 0x7fe79400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4068210 con 0x7fe794004800 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe79400c070 con 0x7fe7940051c0 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe79400c270 con 0x7fe7940039e0 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.375+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b40ba3f0 0x7fe7b40b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe794004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b3200640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7940051c0 0x7fe79400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 -- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7940039e0 msgr2=0x7fe794003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7940039e0 0x7fe794003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 -- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7940051c0 msgr2=0x7fe79400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7940051c0 0x7fe79400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.376+0000 7fe7b1e00640 1 -- 172.21.15.94:0/1994397594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe794003520 con 0x7fe794004800 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.377+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe794004c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40d5720 tx=0x7fe794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5760635514b295cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.377+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c030050 con 0x7fe794004800 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.377+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c003b90 con 0x7fe794004800 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.377+0000 7fe7b1400640 1 -- 172.21.15.94:0/1994397594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c009b50 con 0x7fe794004800 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 -- 172.21.15.94:0/1994397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 msgr2=0x7fe794004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/1994397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe794004c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40d5720 tx=0x7fe794005830 comp rx=0 tx=0).stop 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b3200640 1 -- 172.21.15.94:0/1994397594 reap_dead start 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 -- 172.21.15.94:0/1994397594 shutdown_connections 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 -- 172.21.15.94:0/1994397594 >> 172.21.15.94:0/1994397594 conn(0x7fe7b40d1190 msgr2=0x7fe7b40d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 -- 172.21.15.94:0/1994397594 shutdown_connections 2024-05-27T12:35:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.378+0000 7fe7b8a00640 1 -- 172.21.15.94:0/1994397594 wait complete. 2024-05-27T12:35:39.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.380+0000 7fe7b8a00640 1 Processor -- start 2024-05-27T12:35:39.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.380+0000 7fe7b8a00640 1 -- start start 2024-05-27T12:35:39.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40ca0c0 0x7fe7b40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b415ee60 0x7fe7b415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b415ece0 con 0x7fe794004800 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b40d2df0 con 0x7fe7b40ca0c0 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b8a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b40b6190 con 0x7fe7b415ee60 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56982/0 (socket says 172.21.15.94:56982) 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 -- 172.21.15.94:0/666003000 learned_addr learned my addr 172.21.15.94:0/666003000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40ca0c0 0x7fe7b40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 -- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b415ee60 msgr2=0x7fe7b415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b415ee60 0x7fe7b415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 -- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40ca0c0 msgr2=0x7fe7b40ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40ca0c0 0x7fe7b40ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.381+0000 7fe7b2800640 1 -- 172.21.15.94:0/666003000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b4155c50 con 0x7fe794004800 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7b2800640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400c700 tx=0x7fe7a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4018070 con 0x7fe794004800 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a40074e0 con 0x7fe794004800 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b40caa80 con 0x7fe794004800 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4004a20 con 0x7fe794004800 2024-05-27T12:35:39.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.382+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b4155ef0 con 0x7fe794004800 2024-05-27T12:35:39.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.383+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b4075460 con 0x7fe794004800 2024-05-27T12:35:39.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.385+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7a401b070 con 0x7fe794004800 2024-05-27T12:35:39.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.385+0000 7fe7aaa00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 0x7fe79006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:39.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.386+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7a408b200 con 0x7fe794004800 2024-05-27T12:35:39.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.386+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 0x7fe79006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:39.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.387+0000 7fe7b1e00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 0x7fe79006ee00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe79c035000 tx=0x7fe79c00d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:39.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.390+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7b40b7840 con 0x7fe794004800 2024-05-27T12:35:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.587+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe7b40b9470 con 0x7fe794004800 2024-05-27T12:35:39.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.594+0000 7fe7aaa00640 1 -- 172.21.15.94:0/666003000 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe7a4059610 con 0x7fe794004800 2024-05-27T12:35:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 msgr2=0x7fe79006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 0x7fe79006ee00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe79c035000 tx=0x7fe79c00d040 comp rx=0 tx=0).stop 2024-05-27T12:35:39.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 msgr2=0x7fe7b40cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:39.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400c700 tx=0x7fe7a400cbc0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 shutdown_connections 2024-05-27T12:35:39.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe79006c940 0x7fe79006ee00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40ca0c0 0x7fe7b40ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7b415ee60 0x7fe7b415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.597+0000 7fe7b8a00640 1 --2- 172.21.15.94:0/666003000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe794004800 0x7fe7b40cd890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.598+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 >> 172.21.15.94:0/666003000 conn(0x7fe7b40d1190 msgr2=0x7fe7b40d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.598+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 shutdown_connections 2024-05-27T12:35:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:39.598+0000 7fe7b8a00640 1 -- 172.21.15.94:0/666003000 wait complete. 2024-05-27T12:35:39.620 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-27T12:35:39.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T12:35:39.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T12:35:39.834 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-27T12:35:39.834 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:39.834 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:39.893 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:39.893 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:39.946 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:39.946 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:39.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:40.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.241+0000 7ffb44000640 1 Processor -- start 2024-05-27T12:35:40.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.242+0000 7ffb44000640 1 -- start start 2024-05-27T12:35:40.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb44000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb44000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c0b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb44000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3c1632f0 con 0x7ffb3c0b3860 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb44000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3c0cf2e0 con 0x7ffb3c15ec00 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb44000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3c1635d0 con 0x7ffb3c0b6370 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb41600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb41600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37556/0 (socket says 172.21.15.94:37556) 2024-05-27T12:35:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.243+0000 7ffb41600640 1 -- 172.21.15.94:0/416305011 learned_addr learned my addr 172.21.15.94:0/416305011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.244+0000 7ffb42000640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c0b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.244+0000 7ffb41600640 1 -- 172.21.15.94:0/416305011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 msgr2=0x7ffb3c0b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.244+0000 7ffb41600640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c0b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.244+0000 7ffb41600640 1 -- 172.21.15.94:0/416305011 >> v1:172.21.15.97:6789/0 conn(0x7ffb3c0b3860 legacy=0x7ffb3c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.244+0000 7ffb41600640 1 -- 172.21.15.94:0/416305011 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c0cf530 con 0x7ffb3c15ec00 2024-05-27T12:35:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.245+0000 7ffb41600640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009870 tx=0x7ffb2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b4c2dad9f9d20e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb3be00640 1 -- 172.21.15.94:0/416305011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c037070 con 0x7ffb3c15ec00 2024-05-27T12:35:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb3be00640 1 -- 172.21.15.94:0/416305011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c032440 con 0x7ffb3c15ec00 2024-05-27T12:35:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb3be00640 1 -- 172.21.15.94:0/416305011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c032810 con 0x7ffb3c15ec00 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 -- 172.21.15.94:0/416305011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c15ec00 msgr2=0x7ffb3c0b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009870 tx=0x7ffb2c02a210 comp rx=0 tx=0).stop 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 -- 172.21.15.94:0/416305011 shutdown_connections 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c0b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 --2- 172.21.15.94:0/416305011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0b3db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.246+0000 7ffb44000640 1 -- 172.21.15.94:0/416305011 >> 172.21.15.94:0/416305011 conn(0x7ffb3c0d1370 msgr2=0x7ffb3c0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.247+0000 7ffb44000640 1 -- 172.21.15.94:0/416305011 shutdown_connections 2024-05-27T12:35:40.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.247+0000 7ffb44000640 1 -- 172.21.15.94:0/416305011 wait complete. 2024-05-27T12:35:40.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.248+0000 7ffb44000640 1 Processor -- start 2024-05-27T12:35:40.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.248+0000 7ffb44000640 1 -- start start 2024-05-27T12:35:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c15efe0 con 0x7ffb3c0b3860 2024-05-27T12:35:40.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c0b3530 con 0x7ffb3c15ec00 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.249+0000 7ffb44000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3c15f160 con 0x7ffb3c0b6370 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37564/0 (socket says 172.21.15.94:37564) 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 -- 172.21.15.94:0/2956168422 learned_addr learned my addr 172.21.15.94:0/2956168422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 -- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 msgr2=0x7ffb3c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 -- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb3c15ec00 msgr2=0x7ffb3c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 -- 172.21.15.94:0/2956168422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c15f840 con 0x7ffb3c0b3860 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.250+0000 7ffb41600640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009870 tx=0x7ffb2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.251+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c041020 con 0x7ffb3c0b3860 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.251+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c030970 con 0x7ffb3c0b3860 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.251+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c037040 con 0x7ffb3c0b3860 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.251+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3c15fb20 con 0x7ffb3c0b3860 2024-05-27T12:35:40.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.251+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3c1600e0 con 0x7ffb3c0b3860 2024-05-27T12:35:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.252+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c06db20 con 0x7ffb3c0b3860 2024-05-27T12:35:40.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.253+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb2c044020 con 0x7ffb3c0b3860 2024-05-27T12:35:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.254+0000 7ffb39600640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 0x7ffb1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.254+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb2c0ac670 con 0x7ffb3c0b3860 2024-05-27T12:35:40.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.255+0000 7ffb40c00640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 0x7ffb1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.256+0000 7ffb40c00640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 0x7ffb1c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c147630 tx=0x7ffb28009250 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.258+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb3c06db20 con 0x7ffb3c0b3860 2024-05-27T12:35:40.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.458+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb3c075460 con 0x7ffb3c0b3860 2024-05-27T12:35:40.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.460+0000 7ffb39600640 1 -- 172.21.15.94:0/2956168422 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7ffb2c04b030 con 0x7ffb3c0b3860 2024-05-27T12:35:40.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:40.460 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:40.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 msgr2=0x7ffb1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 0x7ffb1c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c147630 tx=0x7ffb28009250 comp rx=0 tx=0).stop 2024-05-27T12:35:40.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 msgr2=0x7ffb3c152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009870 tx=0x7ffb2c0046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 shutdown_connections 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ffb1c06c8f0 0x7ffb1c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb3c15ec00 0x7ffb3c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb3c0b6370 0x7ffb3c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 --2- 172.21.15.94:0/2956168422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb3c0b3860 0x7ffb3c152080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 >> 172.21.15.94:0/2956168422 conn(0x7ffb3c0d1370 msgr2=0x7ffb3c0d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 shutdown_connections 2024-05-27T12:35:40.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.463+0000 7ffb44000640 1 -- 172.21.15.94:0/2956168422 wait complete. 2024-05-27T12:35:40.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:40.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:40.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.687+0000 7faaf1e00640 1 Processor -- start 2024-05-27T12:35:40.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.687+0000 7faaf1e00640 1 -- start start 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faaec0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaf1e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faaec06fe70 0x7faaec06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaf1e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaec15bae0 con 0x7faaec06be70 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaf1e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7faaec0cf6e0 con 0x7faaec06fe70 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaf1e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0b3540 con 0x7faaec15b700 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaeb400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7faaec06be70 0x7faaec06de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51320/0 (socket says 172.21.15.94:51320) 2024-05-27T12:35:40.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaeb400640 1 -- 172.21.15.94:0/3508000883 learned_addr learned my addr 172.21.15.94:0/3508000883 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaea000640 1 --2- 172.21.15.94:0/3508000883 >> v2:172.21.15.97:3300/0 conn(0x7faaec06fe70 0x7faaec06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faaeaa00640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faaec0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.689+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813016297 0 0) 0x7faaec15bae0 con 0x7faaec06be70 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faad0005730 con 0x7faaec06be70 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1582626348 0 0) 0x7faad0005730 con 0x7faaec06be70 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 msgr2=0x7faaec0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faaec0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 >> v2:172.21.15.97:3300/0 conn(0x7faaec06fe70 msgr2=0x7faaec06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 --2- 172.21.15.94:0/3508000883 >> v2:172.21.15.97:3300/0 conn(0x7faaec06fe70 0x7faaec06b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0b3820 con 0x7faaec06be70 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7faae40019f0 con 0x7faaec06be70 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 >> v1:172.21.15.94:6789/0 conn(0x7faaec06be70 legacy=0x7faaec06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faad0005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006190 0x7faad0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad0006f50 0x7faad00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0cf6e0 con 0x7faad0006f50 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0b3540 con 0x7faad0006190 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faad000dc80 con 0x7faaec15b700 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.690+0000 7faaeb400640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad0006f50 0x7faad00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeaa00640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faad0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaea000640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006190 0x7faad0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeb400640 1 -- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 msgr2=0x7faad0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeb400640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec15b700 0x7faad0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeb400640 1 -- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006190 msgr2=0x7faad0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeb400640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006190 0x7faad0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaeb400640 1 -- 172.21.15.94:0/3508000883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad0005730 con 0x7faad0006f50 2024-05-27T12:35:40.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.691+0000 7faaea000640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006190 0x7faad0006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:40.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.692+0000 7faaeb400640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad0006f50 0x7faad00073d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faae400a7a0 tx=0x7faad0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58b33d635c686ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.692+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400a3f0 con 0x7faad0006f50 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.692+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae402f070 con 0x7faad0006f50 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.692+0000 7faae9600640 1 -- 172.21.15.94:0/3508000883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae402bc60 con 0x7faad0006f50 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 -- 172.21.15.94:0/3508000883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad0006f50 msgr2=0x7faad00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 --2- 172.21.15.94:0/3508000883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faad0006f50 0x7faad00073d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faae400a7a0 tx=0x7faad0007a40 comp rx=0 tx=0).stop 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaeb400640 1 -- 172.21.15.94:0/3508000883 reap_dead start 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 -- 172.21.15.94:0/3508000883 shutdown_connections 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 -- 172.21.15.94:0/3508000883 >> 172.21.15.94:0/3508000883 conn(0x7faaec0b2240 msgr2=0x7faaec0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 -- 172.21.15.94:0/3508000883 shutdown_connections 2024-05-27T12:35:40.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.693+0000 7faaf1e00640 1 -- 172.21.15.94:0/3508000883 wait complete. 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.695+0000 7faaf1e00640 1 Processor -- start 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.695+0000 7faaf1e00640 1 -- start start 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006f50 0x7faaec1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec0aea20 0x7faaec0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0cf3a0 con 0x7faaec1f8200 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0b3de0 con 0x7faad0006f50 2024-05-27T12:35:40.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaf1e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0b5310 con 0x7faaec0aea20 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37586/0 (socket says 172.21.15.94:37586) 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.696+0000 7faaea000640 1 -- 172.21.15.94:0/2513498193 learned_addr learned my addr 172.21.15.94:0/2513498193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaeaa00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006f50 0x7faaec1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 -- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec0aea20 msgr2=0x7faaec0aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec0aea20 0x7faaec0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 -- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006f50 msgr2=0x7faaec1573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006f50 0x7faaec1573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 -- 172.21.15.94:0/2513498193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec15bd10 con 0x7faaec1f8200 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaea000640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faad8010a00 tx=0x7faad8010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad800f040 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad800eba0 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad801c400 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1f8660 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.697+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec15bfd0 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.699+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec068a70 con 0x7faaec1f8200 2024-05-27T12:35:40.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.699+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faad801c5a0 con 0x7faaec1f8200 2024-05-27T12:35:40.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.700+0000 7faae3400640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:40.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.700+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faad808c240 con 0x7faaec1f8200 2024-05-27T12:35:40.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.701+0000 7faaeaa00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:40.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.702+0000 7faaeaa00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 0x7faacc06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faadc009950 tx=0x7faadc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:40.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.704+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faad8018030 con 0x7faaec1f8200 2024-05-27T12:35:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.906+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faaec15b020 con 0x7faaec1f8200 2024-05-27T12:35:40.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.907+0000 7faae3400640 1 -- 172.21.15.94:0/2513498193 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7faad805a660 con 0x7faaec1f8200 2024-05-27T12:35:40.906 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.909+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 msgr2=0x7faacc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.909+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 0x7faacc06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faadc009950 tx=0x7faadc008040 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.909+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 msgr2=0x7faaec14d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.909+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faad8010a00 tx=0x7faad8010ec0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 shutdown_connections 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7faacc06c8f0 0x7faacc06edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faaec0aea20 0x7faaec0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaec1f8200 0x7faaec14d490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 --2- 172.21.15.94:0/2513498193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad0006f50 0x7faaec1573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 >> 172.21.15.94:0/2513498193 conn(0x7faaec0b2240 msgr2=0x7faaec0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 shutdown_connections 2024-05-27T12:35:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:40.910+0000 7faaf1e00640 1 -- 172.21.15.94:0/2513498193 wait complete. 2024-05-27T12:35:40.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.126+0000 7fc3ed000640 1 Processor -- start 2024-05-27T12:35:41.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.127+0000 7fc3ed000640 1 -- start start 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.128+0000 7fc3ed000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e80d4320 0x7fc3e81697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.128+0000 7fc3ed000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e80b7840 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.128+0000 7fc3ed000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e8067f30 con 0x7fc3e815e900 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.128+0000 7fc3ed000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e8068210 con 0x7fc3e80d4320 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6a00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc3e80ba3f0 0x7fc3e80b6190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51332/0 (socket says 172.21.15.94:51332) 2024-05-27T12:35:41.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6a00640 1 -- 172.21.15.94:0/165176236 learned_addr learned my addr 172.21.15.94:0/165176236 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:41.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e7e00640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e80d4320 0x7fc3e81697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40352654 0 0) 0x7fc3d0001110 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3cc003520 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3395305835 0 0) 0x7fc3cc003520 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e80d4320 msgr2=0x7fc3e81697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e80d4320 0x7fc3e81697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 >> v1:172.21.15.97:6789/0 conn(0x7fc3e815e900 legacy=0x7fc3e8067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e80684f0 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fc3d0001840 con 0x7fc3e80ba3f0 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.129+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 >> v1:172.21.15.94:6789/0 conn(0x7fc3e80ba3f0 legacy=0x7fc3e80b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc004800 0x7fc3cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc0051c0 0x7fc3cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8068210 con 0x7fc3cc0039e0 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc00bf90 con 0x7fc3cc004800 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc00c140 con 0x7fc3cc0051c0 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc004800 0x7fc3cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 -- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc0051c0 msgr2=0x7fc3cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc0051c0 0x7fc3cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 -- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc004800 msgr2=0x7fc3cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc004800 0x7fc3cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e7400640 1 -- 172.21.15.94:0/165176236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc003520 con 0x7fc3cc0039e0 2024-05-27T12:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.130+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc004800 0x7fc3cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.131+0000 7fc3e7400640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3cc003e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8001750 tx=0x7fc3cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bc87e198095b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8001370 con 0x7fc3cc0039e0 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d80052d0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3e6000640 1 -- 172.21.15.94:0/165176236 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802abe0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 -- 172.21.15.94:0/165176236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 msgr2=0x7fc3cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 --2- 172.21.15.94:0/165176236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3cc003e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8001750 tx=0x7fc3cc005830 comp rx=0 tx=0).stop 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3e7e00640 1 -- 172.21.15.94:0/165176236 reap_dead start 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 -- 172.21.15.94:0/165176236 shutdown_connections 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 -- 172.21.15.94:0/165176236 >> 172.21.15.94:0/165176236 conn(0x7fc3e80d1190 msgr2=0x7fc3e80d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 -- 172.21.15.94:0/165176236 shutdown_connections 2024-05-27T12:35:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.132+0000 7fc3ed000640 1 -- 172.21.15.94:0/165176236 wait complete. 2024-05-27T12:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.134+0000 7fc3ed000640 1 Processor -- start 2024-05-27T12:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.134+0000 7fc3ed000640 1 -- start start 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 0x7fc3e81e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e81e7c00 0x7fc3e81e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e815ece0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e80d2df0 con 0x7fc3e81e99f0 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.135+0000 7fc3ed000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e80b68a0 con 0x7fc3e81e7c00 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37606/0 (socket says 172.21.15.94:37606) 2024-05-27T12:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 -- 172.21.15.94:0/523376631 learned_addr learned my addr 172.21.15.94:0/523376631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 0x7fc3e81e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 -- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e81e7c00 msgr2=0x7fc3e81e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7e00640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e81e7c00 0x7fc3e81e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e81e7c00 0x7fc3e81e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 -- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 msgr2=0x7fc3e81e9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 0x7fc3e81e9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 -- 172.21.15.94:0/523376631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e81e8950 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 0x7fc3e81e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.136+0000 7fc3e7400640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.137+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8042070 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.137+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d8005480 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.137+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8037040 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.137+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e81e8c30 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.137+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81e91f0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.138+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8075460 con 0x7fc3cc0039e0 2024-05-27T12:35:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.139+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3d80069d0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.140+0000 7fc3df400640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 0x7fc3c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.140+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3d80ab2d0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.140+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 0x7fc3c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.141+0000 7fc3e6a00640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 0x7fc3c406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8152370 tx=0x7fc3d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.144+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3d8079700 con 0x7fc3cc0039e0 2024-05-27T12:35:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.344+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3e80d4910 con 0x7fc3cc0039e0 2024-05-27T12:35:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.345+0000 7fc3df400640 1 -- 172.21.15.94:0/523376631 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc3d80ab0f0 con 0x7fc3cc0039e0 2024-05-27T12:35:41.344 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.347+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 msgr2=0x7fc3c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.347+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 0x7fc3c406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8152370 tx=0x7fc3d0009040 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 msgr2=0x7fc3e80ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8001120 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 shutdown_connections 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc3c406c920 0x7fc3c406ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e81e99f0 0x7fc3e81e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3e81e7c00 0x7fc3e81e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 --2- 172.21.15.94:0/523376631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3cc0039e0 0x7fc3e80ca140 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 >> 172.21.15.94:0/523376631 conn(0x7fc3e80d1190 msgr2=0x7fc3e80cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 shutdown_connections 2024-05-27T12:35:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.348+0000 7fc3ed000640 1 -- 172.21.15.94:0/523376631 wait complete. 2024-05-27T12:35:41.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:41.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.571+0000 7f89f1000640 1 Processor -- start 2024-05-27T12:35:41.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.572+0000 7f89f1000640 1 -- start start 2024-05-27T12:35:41.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89f1000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f89ec15e900 0x7f89ec067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89f1000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 0x7f89ec0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89f1000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec0b7840 con 0x7f89ec0b66d0 2024-05-27T12:35:41.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89f1000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec0b6b00 con 0x7f89ec15e900 2024-05-27T12:35:41.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89f1000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec067f30 con 0x7f89ec0ba3f0 2024-05-27T12:35:41.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89eb400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f89ec15e900 0x7f89ec067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89eb400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f89ec15e900 0x7f89ec067630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37614/0 (socket says 172.21.15.94:37614) 2024-05-27T12:35:41.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89eb400640 1 -- 172.21.15.94:0/1914491493 learned_addr learned my addr 172.21.15.94:0/1914491493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.574+0000 7f89eb400640 1 -- 172.21.15.94:0/1914491493 >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 msgr2=0x7f89ec0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.573+0000 7f89e3e00640 1 --2- 172.21.15.94:0/1914491493 >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 0x7f89ec0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.574+0000 7f89eb400640 1 --2- 172.21.15.94:0/1914491493 >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 0x7f89ec0b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.574+0000 7f89eb400640 1 -- 172.21.15.94:0/1914491493 >> v1:172.21.15.97:6789/0 conn(0x7f89ec0b66d0 legacy=0x7f89ec162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.574+0000 7f89eb400640 1 -- 172.21.15.94:0/1914491493 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec068210 con 0x7f89ec15e900 2024-05-27T12:35:41.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.574+0000 7f89e3e00640 1 --2- 172.21.15.94:0/1914491493 >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 0x7f89ec0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:41.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.575+0000 7f89eb400640 1 --2- 172.21.15.94:0/1914491493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec067630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0d29c0 tx=0x7f89dc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481a441dc8741995 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.575+0000 7f89eaa00640 1 -- 172.21.15.94:0/1914491493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc037070 con 0x7f89ec15e900 2024-05-27T12:35:41.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.575+0000 7f89eaa00640 1 -- 172.21.15.94:0/1914491493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc032440 con 0x7f89ec15e900 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.575+0000 7f89eaa00640 1 -- 172.21.15.94:0/1914491493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc032810 con 0x7f89ec15e900 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 -- 172.21.15.94:0/1914491493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 msgr2=0x7f89ec067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 --2- 172.21.15.94:0/1914491493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec067630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0d29c0 tx=0x7f89dc02a2c0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 -- 172.21.15.94:0/1914491493 shutdown_connections 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 --2- 172.21.15.94:0/1914491493 >> v2:172.21.15.97:3300/0 conn(0x7f89ec0ba3f0 0x7f89ec0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 --2- 172.21.15.94:0/1914491493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec067630 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 -- 172.21.15.94:0/1914491493 >> 172.21.15.94:0/1914491493 conn(0x7f89ec0d1190 msgr2=0x7f89ec0d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 -- 172.21.15.94:0/1914491493 shutdown_connections 2024-05-27T12:35:41.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.576+0000 7f89f1000640 1 -- 172.21.15.94:0/1914491493 wait complete. 2024-05-27T12:35:41.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.578+0000 7f89f1000640 1 Processor -- start 2024-05-27T12:35:41.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.578+0000 7f89f1000640 1 -- start start 2024-05-27T12:35:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec0b66d0 0x7f89ec1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89ec0ba3f0 0x7f89ec158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0d2df0 con 0x7f89ec15e900 2024-05-27T12:35:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0d2b10 con 0x7f89ec0b66d0 2024-05-27T12:35:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89f1000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec15ece0 con 0x7f89ec0ba3f0 2024-05-27T12:35:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37628/0 (socket says 172.21.15.94:37628) 2024-05-27T12:35:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 -- 172.21.15.94:0/1925658377 learned_addr learned my addr 172.21.15.94:0/1925658377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89eb400640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec0b66d0 0x7f89ec1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 -- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89ec0ba3f0 msgr2=0x7f89ec158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89ec0ba3f0 0x7f89ec158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 -- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec0b66d0 msgr2=0x7f89ec1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec0b66d0 0x7f89ec1489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.579+0000 7f89ebe00640 1 -- 172.21.15.94:0/1925658377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec156580 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89ebe00640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89e400c7b0 tx=0x7f89e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4004b60 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e4004d00 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400aa90 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec156860 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.580+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec156e20 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.581+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec075460 con 0x7f89ec15e900 2024-05-27T12:35:41.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.582+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89e400ac30 con 0x7f89ec15e900 2024-05-27T12:35:41.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.583+0000 7f89e3400640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:41.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.583+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89e408be30 con 0x7f89ec15e900 2024-05-27T12:35:41.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.584+0000 7f89eb400640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:41.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.585+0000 7f89eb400640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 0x7f89c806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89dc034000 tx=0x7f89dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.588+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89e4020070 con 0x7f89ec15e900 2024-05-27T12:35:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.788+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f89ec06bbe0 con 0x7f89ec15e900 2024-05-27T12:35:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.789+0000 7f89e3400640 1 -- 172.21.15.94:0/1925658377 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f89e405a250 con 0x7f89ec15e900 2024-05-27T12:35:41.788 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 msgr2=0x7f89c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 0x7f89c806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89dc034000 tx=0x7f89dc02e000 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 msgr2=0x7f89ec158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f89e400c7b0 tx=0x7f89e400cc70 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 shutdown_connections 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f89c806c8f0 0x7f89c806edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89ec15e900 0x7f89ec158640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89ec0ba3f0 0x7f89ec158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 --2- 172.21.15.94:0/1925658377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec0b66d0 0x7f89ec1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 >> 172.21.15.94:0/1925658377 conn(0x7f89ec0d1190 msgr2=0x7f89ec0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.791+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 shutdown_connections 2024-05-27T12:35:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:41.792+0000 7f89f1000640 1 -- 172.21.15.94:0/1925658377 wait complete. 2024-05-27T12:35:41.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T12:35:42.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.011+0000 7f4855400640 1 Processor -- start 2024-05-27T12:35:42.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.011+0000 7f4855400640 1 -- start start 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f4855400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48500d1770 0x7f48500cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f4855400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48500d2a80 con 0x7f4850162170 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f4855400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48500d2d20 con 0x7f48500b51d0 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f4855400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f48500d3650 con 0x7f48500d1770 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f484fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f4850162170 0x7f48500d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51334/0 (socket says 172.21.15.94:51334) 2024-05-27T12:35:42.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f484fe00640 1 -- 172.21.15.94:0/3879618463 learned_addr learned my addr 172.21.15.94:0/3879618463 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512051996 0 0) 0x7f48500d2a80 con 0x7f4850162170 2024-05-27T12:35:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4834003520 con 0x7f4850162170 2024-05-27T12:35:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.013+0000 7f484f400640 1 --2- 172.21.15.94:0/3879618463 >> v2:172.21.15.97:3300/0 conn(0x7f48500d1770 0x7f48500cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1982065956 0 0) 0x7f4834003520 con 0x7f4850162170 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 >> v2:172.21.15.97:3300/0 conn(0x7f48500d1770 msgr2=0x7f48500cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 --2- 172.21.15.94:0/3879618463 >> v2:172.21.15.97:3300/0 conn(0x7f48500d1770 0x7f48500cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 >> v1:172.21.15.97:6789/0 conn(0x7f48500b51d0 legacy=0x7f48500cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48500d3000 con 0x7f4850162170 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f4840001690 con 0x7f4850162170 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 >> v1:172.21.15.94:6789/0 conn(0x7f4850162170 legacy=0x7f48500d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 0x7f4834003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834004800 0x7f4834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834005290 0x7f483400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48500d3650 con 0x7f4834003a00 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f483400c1e0 con 0x7f4834005290 2024-05-27T12:35:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f483400c3e0 con 0x7f4834004800 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.014+0000 7f484fe00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 0x7f4834003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f484fe00640 1 -- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834004800 msgr2=0x7f4834004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f484fe00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834004800 0x7f4834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f484fe00640 1 -- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834005290 msgr2=0x7f483400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f484fe00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4834005290 0x7f483400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f484fe00640 1 -- 172.21.15.94:0/3879618463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4834003520 con 0x7f4834003a00 2024-05-27T12:35:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.015+0000 7f4854a00640 1 -- 172.21.15.94:0/3879618463 reap_dead start 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f484fe00640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 0x7f4834003e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48500d2c50 tx=0x7f4834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c236878c8b236bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840007ca0 con 0x7f4834003a00 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4840005af0 con 0x7f4834003a00 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f484ea00640 1 -- 172.21.15.94:0/3879618463 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840007070 con 0x7f4834003a00 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f4855400640 1 -- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 msgr2=0x7f4834003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f4855400640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 0x7f4834003e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48500d2c50 tx=0x7f4834005830 comp rx=0 tx=0).stop 2024-05-27T12:35:42.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f4855400640 1 -- 172.21.15.94:0/3879618463 shutdown_connections 2024-05-27T12:35:42.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f4855400640 1 --2- 172.21.15.94:0/3879618463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4834003a00 0x7f4834003e00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.016+0000 7f4855400640 1 -- 172.21.15.94:0/3879618463 >> 172.21.15.94:0/3879618463 conn(0x7f485008bfc0 msgr2=0x7f48500d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:42.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.017+0000 7f4855400640 1 -- 172.21.15.94:0/3879618463 shutdown_connections 2024-05-27T12:35:42.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.017+0000 7f4855400640 1 -- 172.21.15.94:0/3879618463 wait complete. 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.018+0000 7f4855400640 1 Processor -- start 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.018+0000 7f4855400640 1 -- start start 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834003a00 0x7f485006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48500b8e50 0x7f48500b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48500b3660 con 0x7f485015cc40 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48500b34e0 con 0x7f48500b8e50 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.019+0000 7f4855400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f48500b37e0 con 0x7f4834003a00 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37650/0 (socket says 172.21.15.94:37650) 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 -- 172.21.15.94:0/3664666481 learned_addr learned my addr 172.21.15.94:0/3664666481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 -- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834003a00 msgr2=0x7f485006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834003a00 0x7f485006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f4854a00640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48500b8e50 0x7f48500b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 -- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48500b8e50 msgr2=0x7f48500b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48500b8e50 0x7f48500b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 -- 172.21.15.94:0/3664666481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48500c93d0 con 0x7f485015cc40 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.020+0000 7f484f400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d580 tx=0x7f483c00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.021+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c0125f0 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.021+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c019070 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.021+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48500c96b0 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.021+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c0129f0 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.021+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48500ca190 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.022+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850075460 con 0x7f485015cc40 2024-05-27T12:35:42.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.023+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f483c012b90 con 0x7f485015cc40 2024-05-27T12:35:42.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.024+0000 7f4847e00640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.024+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f483c08be30 con 0x7f485015cc40 2024-05-27T12:35:42.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.025+0000 7f484fe00640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.026+0000 7f484fe00640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 0x7f483006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4840032000 tx=0x7f4840009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.029+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f483c05a1d0 con 0x7f485015cc40 2024-05-27T12:35:42.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.243+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48500b9530 con 0x7f485015cc40 2024-05-27T12:35:42.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.244+0000 7f4847e00640 1 -- 172.21.15.94:0/3664666481 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f483c059ad0 con 0x7f485015cc40 2024-05-27T12:35:42.243 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-05-27T12:35:42.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 msgr2=0x7f483006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 0x7f483006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4840032000 tx=0x7f4840009030 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 msgr2=0x7f485006daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d580 tx=0x7f483c00da40 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 shutdown_connections 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f483006c920 0x7f483006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48500b8e50 0x7f48500b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f485015cc40 0x7f485006daf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 --2- 172.21.15.94:0/3664666481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834003a00 0x7f485006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 >> 172.21.15.94:0/3664666481 conn(0x7f485008bfc0 msgr2=0x7f48500d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 shutdown_connections 2024-05-27T12:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.247+0000 7f4855400640 1 -- 172.21.15.94:0/3664666481 wait complete. 2024-05-27T12:35:42.264 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T12:35:42.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:42.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-27T12:35:42.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:35:42.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:42.341 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T12:35:42.397 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:42.397 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:42.454 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:42.454 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:42.510 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:42.510 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:42.563 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:42.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.772+0000 7f1663e00640 1 Processor -- start 2024-05-27T12:35:42.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.772+0000 7f1663e00640 1 -- start start 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1663e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1664145cf0 0x7f1664160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1663e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640ceae0 con 0x7f16641460d0 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1663e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16641602c0 con 0x7f1664071d60 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1663e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f16641605a0 con 0x7f1664145cf0 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f165a000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f16641460d0 0x7f16640ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51350/0 (socket says 172.21.15.94:51350) 2024-05-27T12:35:42.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f165a000640 1 -- 172.21.15.94:0/2871713207 learned_addr learned my addr 172.21.15.94:0/2871713207 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702933966 0 0) 0x7f16640ceae0 con 0x7f16641460d0 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1648003520 con 0x7f16641460d0 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.774+0000 7f1662a00640 1 --2- 172.21.15.94:0/2871713207 >> v2:172.21.15.97:3300/0 conn(0x7f1664145cf0 0x7f1664160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2184779069 0 0) 0x7f1648003520 con 0x7f16641460d0 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 >> v2:172.21.15.97:3300/0 conn(0x7f1664145cf0 msgr2=0x7f1664160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 --2- 172.21.15.94:0/2871713207 >> v2:172.21.15.97:3300/0 conn(0x7f1664145cf0 0x7f1664160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 >> v1:172.21.15.97:6789/0 conn(0x7f1664071d60 legacy=0x7f16640cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1664160880 con 0x7f16641460d0 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f164c001670 con 0x7f16641460d0 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 >> v1:172.21.15.94:6789/0 conn(0x7f16641460d0 legacy=0x7f16640ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1648003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1648004800 0x7f1648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1648005210 0x7f164800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16641605a0 con 0x7f1648003a00 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f164800c140 con 0x7f1648004800 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f164800c340 con 0x7f1648005210 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.775+0000 7f1662a00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1648003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f165a000640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1648004800 0x7f1648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1662a00640 1 -- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1648005210 msgr2=0x7f164800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1662a00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1648005210 0x7f164800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1662a00640 1 -- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1648004800 msgr2=0x7f1648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:42.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1662a00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1648004800 0x7f1648004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1662a00640 1 -- 172.21.15.94:0/2871713207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1648003520 con 0x7f1648003a00 2024-05-27T12:35:42.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.776+0000 7f1663400640 1 -- 172.21.15.94:0/2871713207 reap_dead start 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.777+0000 7f1662a00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1648003e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f165400ba00 tx=0x7f1648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dce3148610edb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.777+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654030070 con 0x7f1648003a00 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.777+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165402ada0 con 0x7f1648003a00 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.777+0000 7f1662000640 1 -- 172.21.15.94:0/2871713207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165400b600 con 0x7f1648003a00 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.777+0000 7f1663e00640 1 -- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 msgr2=0x7f1648003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1648003e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f165400ba00 tx=0x7f1648005830 comp rx=0 tx=0).stop 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 -- 172.21.15.94:0/2871713207 shutdown_connections 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 --2- 172.21.15.94:0/2871713207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1648003e00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 -- 172.21.15.94:0/2871713207 >> 172.21.15.94:0/2871713207 conn(0x7f16640b2160 msgr2=0x7f16640b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:42.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 -- 172.21.15.94:0/2871713207 shutdown_connections 2024-05-27T12:35:42.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.778+0000 7f1663e00640 1 -- 172.21.15.94:0/2871713207 wait complete. 2024-05-27T12:35:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.779+0000 7f1663e00640 1 Processor -- start 2024-05-27T12:35:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 -- start start 2024-05-27T12:35:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166413f2c0 0x7f166413d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f166413dd10 0x7f1664143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f166406ec80 con 0x7f1648003a00 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.780+0000 7f1663e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f166406eb00 con 0x7f166413f2c0 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1663e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1664160da0 con 0x7f166413dd10 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37662/0 (socket says 172.21.15.94:37662) 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 -- 172.21.15.94:0/3110614263 learned_addr learned my addr 172.21.15.94:0/3110614263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f165a000640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166413f2c0 0x7f166413d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 -- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f166413dd10 msgr2=0x7f1664143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f166413dd10 0x7f1664143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 -- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166413f2c0 msgr2=0x7f166413d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166413f2c0 0x7f166413d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.781+0000 7f1662a00640 1 -- 172.21.15.94:0/3110614263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166413e220 con 0x7f1648003a00 2024-05-27T12:35:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f1662a00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f165400bb20 tx=0x7f1654004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654043020 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165403f680 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166413e500 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165400b600 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.782+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166413eac0 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.783+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16640cfc00 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.784+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1654046020 con 0x7f1648003a00 2024-05-27T12:35:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.785+0000 7f165b400640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.785+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f16540ac5c0 con 0x7f1648003a00 2024-05-27T12:35:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.786+0000 7f165a000640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.787+0000 7f165a000640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1664153a50 tx=0x7f164c000f50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:42.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.789+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16640cfc00 con 0x7f1648003a00 2024-05-27T12:35:42.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.989+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1664075460 con 0x7f1648003a00 2024-05-27T12:35:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.992+0000 7f165b400640 1 -- 172.21.15.94:0/3110614263 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f165404d030 con 0x7f1648003a00 2024-05-27T12:35:42.992 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:42.992 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:00.929324+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.994+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 msgr2=0x7f164006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.994+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1664153a50 tx=0x7f164c000f50 comp rx=0 tx=0).stop 2024-05-27T12:35:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 msgr2=0x7f1664152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f165400bb20 tx=0x7f1654004730 comp rx=0 tx=0).stop 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 shutdown_connections 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166413f2c0 0x7f166413d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f166413dd10 0x7f1664143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 --2- 172.21.15.94:0/3110614263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1648003a00 0x7f1664152cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 >> 172.21.15.94:0/3110614263 conn(0x7f16640b2160 msgr2=0x7f16640af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 shutdown_connections 2024-05-27T12:35:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:42.995+0000 7f1663e00640 1 -- 172.21.15.94:0/3110614263 wait complete. 2024-05-27T12:35:43.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:43.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:43.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.223+0000 7f0b96000640 1 Processor -- start 2024-05-27T12:35:43.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.224+0000 7f0b96000640 1 -- start start 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.225+0000 7f0b96000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b900d4320 0x7f0b901697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.225+0000 7f0b96000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b900b7840 con 0x7f0b9015e900 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.225+0000 7f0b96000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b90067f30 con 0x7f0b900ba3f0 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.225+0000 7f0b96000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0b90068210 con 0x7f0b900d4320 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.225+0000 7f0b94c00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f0b9015e900 0x7f0b90067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51352/0 (socket says 172.21.15.94:51352) 2024-05-27T12:35:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b94c00640 1 -- 172.21.15.94:0/2184713387 learned_addr learned my addr 172.21.15.94:0/2184713387 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b95600640 1 --2- 172.21.15.94:0/2184713387 >> v2:172.21.15.97:3300/0 conn(0x7f0b900d4320 0x7f0b901697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819215709 0 0) 0x7f0b80001490 con 0x7f0b9015e900 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b74003520 con 0x7f0b9015e900 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3309556333 0 0) 0x7f0b74003520 con 0x7f0b9015e900 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 >> v2:172.21.15.97:3300/0 conn(0x7f0b900d4320 msgr2=0x7f0b901697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 --2- 172.21.15.94:0/2184713387 >> v2:172.21.15.97:3300/0 conn(0x7f0b900d4320 0x7f0b901697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.226+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 >> v1:172.21.15.97:6789/0 conn(0x7f0b900ba3f0 legacy=0x7f0b900b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b900684f0 con 0x7f0b9015e900 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f0b80001bc0 con 0x7f0b9015e900 2024-05-27T12:35:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 >> v1:172.21.15.94:6789/0 conn(0x7f0b9015e900 legacy=0x7f0b90067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b740039b0 0x7f0b74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90068210 con 0x7f0b74004800 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7400c070 con 0x7f0b740039b0 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7400c270 con 0x7f0b740051c0 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b95600640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b94c00640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b740039b0 0x7f0b74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 -- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b740051c0 msgr2=0x7f0b7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 -- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b740039b0 msgr2=0x7f0b74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b740039b0 0x7f0b74003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b8fe00640 1 -- 172.21.15.94:0/2184713387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74003520 con 0x7f0b74004800 2024-05-27T12:35:43.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.227+0000 7f0b95600640 1 -- 172.21.15.94:0/2184713387 reap_dead start 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.228+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b78001750 tx=0x7f0b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128c42012251ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78001370 con 0x7f0b74004800 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b780052d0 con 0x7f0b74004800 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b8f400640 1 -- 172.21.15.94:0/2184713387 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7802abc0 con 0x7f0b74004800 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 -- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 msgr2=0x7f0b74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b78001750 tx=0x7f0b74005830 comp rx=0 tx=0).stop 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 -- 172.21.15.94:0/2184713387 shutdown_connections 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 --2- 172.21.15.94:0/2184713387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 -- 172.21.15.94:0/2184713387 >> 172.21.15.94:0/2184713387 conn(0x7f0b900d1190 msgr2=0x7f0b900d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.229+0000 7f0b96000640 1 -- 172.21.15.94:0/2184713387 shutdown_connections 2024-05-27T12:35:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.230+0000 7f0b96000640 1 -- 172.21.15.94:0/2184713387 wait complete. 2024-05-27T12:35:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.231+0000 7f0b96000640 1 Processor -- start 2024-05-27T12:35:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.231+0000 7f0b96000640 1 -- start start 2024-05-27T12:35:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b74004800 0x7f0b901f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b901f83c0 0x7f0b90151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9015ece0 con 0x7f0b901553f0 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b900d2df0 con 0x7f0b74004800 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.232+0000 7f0b96000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f0b90067630 con 0x7f0b901f83c0 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37678/0 (socket says 172.21.15.94:37678) 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 -- 172.21.15.94:0/808052511 learned_addr learned my addr 172.21.15.94:0/808052511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b94c00640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b74004800 0x7f0b901f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b901f83c0 0x7f0b90151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 -- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b901f83c0 msgr2=0x7f0b90151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b901f83c0 0x7f0b90151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 -- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b74004800 msgr2=0x7f0b901f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b74004800 0x7f0b901f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.233+0000 7f0b95600640 1 -- 172.21.15.94:0/808052511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b901f8b30 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b95600640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b9008cc30 tx=0x7f0b88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88003610 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8801b070 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88003a30 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b901f8e10 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.234+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901f8840 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.235+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90160e20 con 0x7f0b901553f0 2024-05-27T12:35:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.236+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b88003bd0 con 0x7f0b901553f0 2024-05-27T12:35:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.237+0000 7f0b8cc00640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.237+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b8808bfe0 con 0x7f0b901553f0 2024-05-27T12:35:43.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.238+0000 7f0b94c00640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.239+0000 7f0b94c00640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b80002230 tx=0x7f0b80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.242+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b8805a370 con 0x7f0b901553f0 2024-05-27T12:35:43.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.440+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b90075460 con 0x7f0b901553f0 2024-05-27T12:35:43.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.441+0000 7f0b8cc00640 1 -- 172.21.15.94:0/808052511 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0b88060030 con 0x7f0b901553f0 2024-05-27T12:35:43.440 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 msgr2=0x7f0b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b80002230 tx=0x7f0b80009040 comp rx=0 tx=0).stop 2024-05-27T12:35:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 msgr2=0x7f0b900cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b9008cc30 tx=0x7f0b88009f90 comp rx=0 tx=0).stop 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 shutdown_connections 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b901553f0 0x7f0b900cda90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b901f83c0 0x7f0b90151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 --2- 172.21.15.94:0/808052511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b74004800 0x7f0b901f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 >> 172.21.15.94:0/808052511 conn(0x7f0b900d1190 msgr2=0x7f0b90154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 shutdown_connections 2024-05-27T12:35:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.444+0000 7f0b96000640 1 -- 172.21.15.94:0/808052511 wait complete. 2024-05-27T12:35:43.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:43.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.666+0000 7ff9d6000640 1 Processor -- start 2024-05-27T12:35:43.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.666+0000 7ff9d6000640 1 -- start start 2024-05-27T12:35:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d6000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff9d0072430 0x7ff9d015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9d015d120 0x7ff9d0140a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d6000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d006dc50 con 0x7ff9d013f480 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d6000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d013fcd0 con 0x7ff9d0072430 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d6000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d0140570 con 0x7ff9d015d120 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9cfe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff9d0072430 0x7ff9d015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9cfe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff9d0072430 0x7ff9d015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37690/0 (socket says 172.21.15.94:37690) 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9cfe00640 1 -- 172.21.15.94:0/3624461030 learned_addr learned my addr 172.21.15.94:0/3624461030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.668+0000 7ff9d5600640 1 --2- 172.21.15.94:0/3624461030 >> v2:172.21.15.97:3300/0 conn(0x7ff9d015d120 0x7ff9d0140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.669+0000 7ff9cfe00640 1 -- 172.21.15.94:0/3624461030 >> v2:172.21.15.97:3300/0 conn(0x7ff9d015d120 msgr2=0x7ff9d0140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.669+0000 7ff9cfe00640 1 --2- 172.21.15.94:0/3624461030 >> v2:172.21.15.97:3300/0 conn(0x7ff9d015d120 0x7ff9d0140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.669+0000 7ff9cfe00640 1 -- 172.21.15.94:0/3624461030 >> v1:172.21.15.97:6789/0 conn(0x7ff9d013f480 legacy=0x7ff9d0071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.669+0000 7ff9cfe00640 1 -- 172.21.15.94:0/3624461030 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d015d950 con 0x7ff9d0072430 2024-05-27T12:35:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.670+0000 7ff9cfe00640 1 --2- 172.21.15.94:0/3624461030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0072430 0x7ff9d015cbe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b6800 tx=0x7ff9bc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731d41d53f3d97a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.670+0000 7ff9cf400640 1 -- 172.21.15.94:0/3624461030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037070 con 0x7ff9d0072430 2024-05-27T12:35:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.670+0000 7ff9cf400640 1 -- 172.21.15.94:0/3624461030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc032440 con 0x7ff9d0072430 2024-05-27T12:35:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.670+0000 7ff9cf400640 1 -- 172.21.15.94:0/3624461030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc032810 con 0x7ff9d0072430 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 -- 172.21.15.94:0/3624461030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0072430 msgr2=0x7ff9d015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 --2- 172.21.15.94:0/3624461030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0072430 0x7ff9d015cbe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b6800 tx=0x7ff9bc02a200 comp rx=0 tx=0).stop 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 -- 172.21.15.94:0/3624461030 shutdown_connections 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 --2- 172.21.15.94:0/3624461030 >> v2:172.21.15.97:3300/0 conn(0x7ff9d015d120 0x7ff9d0140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 --2- 172.21.15.94:0/3624461030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d0072430 0x7ff9d015cbe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 -- 172.21.15.94:0/3624461030 >> 172.21.15.94:0/3624461030 conn(0x7ff9d00b6230 msgr2=0x7ff9d00b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 -- 172.21.15.94:0/3624461030 shutdown_connections 2024-05-27T12:35:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.671+0000 7ff9d6000640 1 -- 172.21.15.94:0/3624461030 wait complete. 2024-05-27T12:35:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.673+0000 7ff9d6000640 1 Processor -- start 2024-05-27T12:35:43.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.673+0000 7ff9d6000640 1 -- start start 2024-05-27T12:35:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 0x7ff9d015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9d013f480 0x7ff9d013c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d013f860 con 0x7ff9d015d120 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0140200 con 0x7ff9d013f480 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d6000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d013f9e0 con 0x7ff9d0072430 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37706/0 (socket says 172.21.15.94:37706) 2024-05-27T12:35:43.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d5600640 1 -- 172.21.15.94:0/2864895029 learned_addr learned my addr 172.21.15.94:0/2864895029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.674+0000 7ff9d4c00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 0x7ff9d015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 -- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 msgr2=0x7ff9d015b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9cfe00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9d013f480 0x7ff9d013c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 0x7ff9d015b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 -- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9d013f480 msgr2=0x7ff9d013c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9d013f480 0x7ff9d013c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 -- 172.21.15.94:0/2864895029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d0158900 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d4c00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 0x7ff9d015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d5600640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8007e40 tx=0x7ff9c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8014620 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c8019070 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d0158be0 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8014a20 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.675+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d015bf20 con 0x7ff9d015d120 2024-05-27T12:35:43.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.676+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d0075460 con 0x7ff9d015d120 2024-05-27T12:35:43.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.677+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9c8014bc0 con 0x7ff9d015d120 2024-05-27T12:35:43.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.678+0000 7ff9ccc00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:43.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.678+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9c808c4b0 con 0x7ff9d015d120 2024-05-27T12:35:43.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.679+0000 7ff9d4c00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:43.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.680+0000 7ff9d4c00640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 0x7ff9b006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4003eb0 tx=0x7ff9c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:43.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.683+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9c805a840 con 0x7ff9d015d120 2024-05-27T12:35:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.882+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9d0072f30 con 0x7ff9d015d120 2024-05-27T12:35:43.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.883+0000 7ff9ccc00640 1 -- 172.21.15.94:0/2864895029 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff9c805a140 con 0x7ff9d015d120 2024-05-27T12:35:43.882 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 msgr2=0x7ff9b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 0x7ff9b006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4003eb0 tx=0x7ff9c4009040 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 msgr2=0x7ff9d013cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8007e40 tx=0x7ff9c800e4b0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 shutdown_connections 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff9b006c8f0 0x7ff9b006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9d013f480 0x7ff9d013c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d015d120 0x7ff9d013cde0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 --2- 172.21.15.94:0/2864895029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9d0072430 0x7ff9d015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 >> 172.21.15.94:0/2864895029 conn(0x7ff9d00b6230 msgr2=0x7ff9d015e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 shutdown_connections 2024-05-27T12:35:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:43.885+0000 7ff9d6000640 1 -- 172.21.15.94:0/2864895029 wait complete. 2024-05-27T12:35:43.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:44.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.106+0000 7f7910000640 1 Processor -- start 2024-05-27T12:35:44.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.107+0000 7f7910000640 1 -- start start 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.108+0000 7f7910000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f790806bf80 0x7f79080d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.108+0000 7f7910000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f7908145770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.108+0000 7f7910000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080ce090 con 0x7f790806bba0 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.108+0000 7f7910000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f79080d2480 con 0x7f790806bf80 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.108+0000 7f7910000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f79080ce370 con 0x7f79080d2010 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f790d600640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f790806bba0 0x7f79080b1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51364/0 (socket says 172.21.15.94:51364) 2024-05-27T12:35:44.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f790d600640 1 -- 172.21.15.94:0/2866047534 learned_addr learned my addr 172.21.15.94:0/2866047534 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825485184 0 0) 0x7f79080ce090 con 0x7f790806bba0 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f790e000640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f7908145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f790cc00640 1 --2- 172.21.15.94:0/2866047534 >> v2:172.21.15.97:3300/0 conn(0x7f790806bf80 0x7f79080d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f0003520 con 0x7f790806bba0 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132655452 0 0) 0x7f78f0003520 con 0x7f790806bba0 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 msgr2=0x7f7908145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f7908145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 >> v2:172.21.15.97:3300/0 conn(0x7f790806bf80 msgr2=0x7f79080d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 --2- 172.21.15.94:0/2866047534 >> v2:172.21.15.97:3300/0 conn(0x7f790806bf80 0x7f79080d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.109+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79080ce650 con 0x7f790806bba0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f78fc0019f0 con 0x7f790806bba0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 >> v1:172.21.15.94:6789/0 conn(0x7f790806bba0 legacy=0x7f79080b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f78f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f78f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0004d40 0x7f78f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79080ce370 con 0x7f78f00039b0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000baa0 con 0x7f78f0004d40 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f78f000bca0 con 0x7f79080d2010 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f78f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790cc00640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f78f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 -- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 msgr2=0x7f78f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080d2010 0x7f78f0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790e000640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0004d40 0x7f78f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 -- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0004d40 msgr2=0x7f78f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0004d40 0x7f78f00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.110+0000 7f790d600640 1 -- 172.21.15.94:0/2866047534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0003520 con 0x7f78f00039b0 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.111+0000 7f790d600640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f78f0003dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00a920 tx=0x7f78f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7adac6ef7c1e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00a4c0 con 0x7f78f00039b0 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78fc02f070 con 0x7f78f00039b0 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7907e00640 1 -- 172.21.15.94:0/2866047534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02bd00 con 0x7f78f00039b0 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 -- 172.21.15.94:0/2866047534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 msgr2=0x7f78f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 --2- 172.21.15.94:0/2866047534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f78f0003dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00a920 tx=0x7f78f0005830 comp rx=0 tx=0).stop 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f790e000640 1 -- 172.21.15.94:0/2866047534 reap_dead start 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 -- 172.21.15.94:0/2866047534 shutdown_connections 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 -- 172.21.15.94:0/2866047534 >> 172.21.15.94:0/2866047534 conn(0x7f79080734b0 msgr2=0x7f790809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 -- 172.21.15.94:0/2866047534 shutdown_connections 2024-05-27T12:35:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.112+0000 7f7910000640 1 -- 172.21.15.94:0/2866047534 wait complete. 2024-05-27T12:35:44.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.114+0000 7f7910000640 1 Processor -- start 2024-05-27T12:35:44.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.114+0000 7f7910000640 1 -- start start 2024-05-27T12:35:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080b4300 0x7f79080cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79080c7620 0x7f79080c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79080b2650 con 0x7f78f00039b0 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79080cdd60 con 0x7f79080c7620 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.115+0000 7f7910000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7908145770 con 0x7f79080b4300 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37716/0 (socket says 172.21.15.94:37716) 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 -- 172.21.15.94:0/187978877 learned_addr learned my addr 172.21.15.94:0/187978877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790cc00640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080b4300 0x7f79080cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790e000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79080c7620 0x7f79080c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 -- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080b4300 msgr2=0x7f79080cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080b4300 0x7f79080cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 -- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79080c7620 msgr2=0x7f79080c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79080c7620 0x7f79080c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.116+0000 7f790d600640 1 -- 172.21.15.94:0/187978877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790813d510 con 0x7f78f00039b0 2024-05-27T12:35:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f790d600640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00a920 tx=0x7f79080ce260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc002000 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78fc004d60 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc034670 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79080b48c0 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.117+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79080b4e80 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.118+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7908068a70 con 0x7f78f00039b0 2024-05-27T12:35:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.119+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78fc034810 con 0x7f78f00039b0 2024-05-27T12:35:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.120+0000 7f7905600640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.120+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78fc0abff0 con 0x7f78f00039b0 2024-05-27T12:35:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.121+0000 7f790cc00640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.122+0000 7f790cc00640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 0x7f78e806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7908070f40 tx=0x7f78f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.124+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78fc07a380 con 0x7f78f00039b0 2024-05-27T12:35:44.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.326+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f790806c7f0 con 0x7f78f00039b0 2024-05-27T12:35:44.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.327+0000 7f7905600640 1 -- 172.21.15.94:0/187978877 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f78fc079c80 con 0x7f78f00039b0 2024-05-27T12:35:44.327 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 msgr2=0x7f78e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 0x7f78e806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7908070f40 tx=0x7f78f4008040 comp rx=0 tx=0).stop 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 msgr2=0x7f790813cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00a920 tx=0x7f79080ce260 comp rx=0 tx=0).stop 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 shutdown_connections 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79080c7620 0x7f79080c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79080b4300 0x7f79080cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 --2- 172.21.15.94:0/187978877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78f00039b0 0x7f790813cfd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 >> 172.21.15.94:0/187978877 conn(0x7f79080734b0 msgr2=0x7f790806f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 shutdown_connections 2024-05-27T12:35:44.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.330+0000 7f7910000640 1 -- 172.21.15.94:0/187978877 wait complete. 2024-05-27T12:35:44.348 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.552+0000 7fa63e200640 1 Processor -- start 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.553+0000 7fa63e200640 1 -- start start 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa63e200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa63815cb20 0x7fa6380d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa63e200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6380b1f90 0x7fa6380cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa63e200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa63809b790 con 0x7fa6380b2400 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa63e200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa63809ba70 con 0x7fa63815cb20 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa63e200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa63809bd50 con 0x7fa6380b1f90 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.554+0000 7fa637400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa63815cb20 0x7fa6380d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa63815cb20 0x7fa6380d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37718/0 (socket says 172.21.15.94:37718) 2024-05-27T12:35:44.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 -- 172.21.15.94:0/3408397308 learned_addr learned my addr 172.21.15.94:0/3408397308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:44.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa636a00640 1 --2- 172.21.15.94:0/3408397308 >> v2:172.21.15.97:3300/0 conn(0x7fa6380b1f90 0x7fa6380cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 -- 172.21.15.94:0/3408397308 >> v2:172.21.15.97:3300/0 conn(0x7fa6380b1f90 msgr2=0x7fa6380cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 --2- 172.21.15.94:0/3408397308 >> v2:172.21.15.97:3300/0 conn(0x7fa6380b1f90 0x7fa6380cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 -- 172.21.15.94:0/3408397308 >> v1:172.21.15.97:6789/0 conn(0x7fa6380b2400 legacy=0x7fa6380d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:44.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.555+0000 7fa637400640 1 -- 172.21.15.94:0/3408397308 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380cd320 con 0x7fa63815cb20 2024-05-27T12:35:44.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.556+0000 7fa637400640 1 --2- 172.21.15.94:0/3408397308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa63815cb20 0x7fa6380d12e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6380732a0 tx=0x7fa62802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7ac18a808935d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.556+0000 7fa636000640 1 -- 172.21.15.94:0/3408397308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628037070 con 0x7fa63815cb20 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa636000640 1 -- 172.21.15.94:0/3408397308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62802ae20 con 0x7fa63815cb20 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa636000640 1 -- 172.21.15.94:0/3408397308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628032660 con 0x7fa63815cb20 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 -- 172.21.15.94:0/3408397308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa63815cb20 msgr2=0x7fa6380d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 --2- 172.21.15.94:0/3408397308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa63815cb20 0x7fa6380d12e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6380732a0 tx=0x7fa62802a110 comp rx=0 tx=0).stop 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 -- 172.21.15.94:0/3408397308 shutdown_connections 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 --2- 172.21.15.94:0/3408397308 >> v2:172.21.15.97:3300/0 conn(0x7fa6380b1f90 0x7fa6380cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 --2- 172.21.15.94:0/3408397308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa63815cb20 0x7fa6380d12e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 -- 172.21.15.94:0/3408397308 >> 172.21.15.94:0/3408397308 conn(0x7fa6380b4160 msgr2=0x7fa6380b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 -- 172.21.15.94:0/3408397308 shutdown_connections 2024-05-27T12:35:44.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.557+0000 7fa63e200640 1 -- 172.21.15.94:0/3408397308 wait complete. 2024-05-27T12:35:44.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.559+0000 7fa63e200640 1 Processor -- start 2024-05-27T12:35:44.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.559+0000 7fa63e200640 1 -- start start 2024-05-27T12:35:44.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6380b2400 0x7fa6380b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa63815cb20 0x7fa6381f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380ce1e0 con 0x7fa6380b1f90 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380ce060 con 0x7fa6380b2400 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.560+0000 7fa63e200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380ce360 con 0x7fa63815cb20 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37720/0 (socket says 172.21.15.94:37720) 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 -- 172.21.15.94:0/1698201403 learned_addr learned my addr 172.21.15.94:0/1698201403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa636a00640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6380b2400 0x7fa6380b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 -- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa63815cb20 msgr2=0x7fa6381f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637e00640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa63815cb20 0x7fa6381f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa63815cb20 0x7fa6381f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 -- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6380b2400 msgr2=0x7fa6380b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6380b2400 0x7fa6380b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.561+0000 7fa637400640 1 -- 172.21.15.94:0/1698201403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6381f8bd0 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa637400640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa628000c00 tx=0x7fa6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628041070 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62803c670 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381f8eb0 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628037040 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.562+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381f50a0 con 0x7fa6380b1f90 2024-05-27T12:35:44.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.563+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638068a70 con 0x7fa6380b1f90 2024-05-27T12:35:44.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.564+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa628044070 con 0x7fa6380b1f90 2024-05-27T12:35:44.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.565+0000 7fa62f400640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:44.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.565+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6280ac460 con 0x7fa6380b1f90 2024-05-27T12:35:44.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.566+0000 7fa636a00640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:44.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.567+0000 7fa636a00640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa6380c7950 tx=0x7fa620003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.570+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa62807a910 con 0x7fa6380b1f90 2024-05-27T12:35:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.780+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa6380b4e80 con 0x7fa6380b1f90 2024-05-27T12:35:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.781+0000 7fa62f400640 1 -- 172.21.15.94:0/1698201403 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fa62802ae20 con 0x7fa6380b1f90 2024-05-27T12:35:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-05-27T12:35:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 msgr2=0x7fa61406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa6380c7950 tx=0x7fa620003990 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 msgr2=0x7fa6381f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa628000c00 tx=0x7fa6280046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 shutdown_connections 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6380b2400 0x7fa6380b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa63815cb20 0x7fa6381f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 --2- 172.21.15.94:0/1698201403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6380b1f90 0x7fa6381f55c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 >> 172.21.15.94:0/1698201403 conn(0x7fa6380b4160 msgr2=0x7fa6380af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 shutdown_connections 2024-05-27T12:35:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:44.784+0000 7fa63e200640 1 -- 172.21.15.94:0/1698201403 wait complete. 2024-05-27T12:35:44.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-27T12:35:44.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:44.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-27T12:35:44.907 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:35:44.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:44.907 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-27T12:35:44.973 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:44.973 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:45.031 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:45.031 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:45.082 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:35:45.083 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:45.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:45.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.335+0000 7f4083200640 1 Processor -- start 2024-05-27T12:35:45.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.335+0000 7f4083200640 1 -- start start 2024-05-27T12:35:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4083200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 0x7f407c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4083200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f407c06eed0 0x7f407c06f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4083200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f407c13e9e0 con 0x7f407c0cd480 2024-05-27T12:35:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4083200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f407c15ba50 con 0x7f407c06eed0 2024-05-27T12:35:45.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4083200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f407c15bd10 con 0x7f407c13e600 2024-05-27T12:35:45.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4082800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f407c0cd480 0x7f407c0cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51372/0 (socket says 172.21.15.94:51372) 2024-05-27T12:35:45.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4082800640 1 -- 172.21.15.94:0/1078457191 learned_addr learned my addr 172.21.15.94:0/1078457191 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897734260 0 0) 0x7f4078001490 con 0x7f407c0cd480 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4081400640 1 --2- 172.21.15.94:0/1078457191 >> v2:172.21.15.97:3300/0 conn(0x7f407c06eed0 0x7f407c06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.336+0000 7f4081e00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 0x7f407c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4064003520 con 0x7f407c0cd480 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3462608044 0 0) 0x7f4064003520 con 0x7f407c0cd480 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 msgr2=0x7f407c071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 0x7f407c071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 >> v2:172.21.15.97:3300/0 conn(0x7f407c06eed0 msgr2=0x7f407c06f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 --2- 172.21.15.94:0/1078457191 >> v2:172.21.15.97:3300/0 conn(0x7f407c06eed0 0x7f407c06f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c15bff0 con 0x7f407c0cd480 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f4078001bc0 con 0x7f407c0cd480 2024-05-27T12:35:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 >> v1:172.21.15.94:6789/0 conn(0x7f407c0cd480 legacy=0x7f407c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40640039b0 0x7f4064003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 0x7f4064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4064004d40 0x7f40640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f407c15ba50 con 0x7f40640039b0 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f407c15bd10 con 0x7f4064004d40 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f406400b9c0 con 0x7f407c13e600 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.337+0000 7f4081e00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40640039b0 0x7f4064003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4082800640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4064004d40 0x7f40640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4081e00640 1 -- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 msgr2=0x7f4064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4081e00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c13e600 0x7f4064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4081e00640 1 -- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4064004d40 msgr2=0x7f40640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4081e00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4064004d40 0x7f40640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4081e00640 1 -- 172.21.15.94:0/1078457191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4064003520 con 0x7f40640039b0 2024-05-27T12:35:45.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.338+0000 7f4082800640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4064004d40 0x7f40640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.339+0000 7f4081e00640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40640039b0 0x7f4064003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4068009ac0 tx=0x7f4064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5055edc5b1842497 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.339+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068036070 con 0x7f40640039b0 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.339+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4068031400 con 0x7f40640039b0 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.339+0000 7f4080a00640 1 -- 172.21.15.94:0/1078457191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068009690 con 0x7f40640039b0 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 -- 172.21.15.94:0/1078457191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40640039b0 msgr2=0x7f4064003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 --2- 172.21.15.94:0/1078457191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40640039b0 0x7f4064003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4068009ac0 tx=0x7f4064005830 comp rx=0 tx=0).stop 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4082800640 1 -- 172.21.15.94:0/1078457191 reap_dead start 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 -- 172.21.15.94:0/1078457191 shutdown_connections 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 -- 172.21.15.94:0/1078457191 >> 172.21.15.94:0/1078457191 conn(0x7f407c0b2160 msgr2=0x7f407c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:45.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 -- 172.21.15.94:0/1078457191 shutdown_connections 2024-05-27T12:35:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.340+0000 7f4083200640 1 -- 172.21.15.94:0/1078457191 wait complete. 2024-05-27T12:35:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.342+0000 7f4083200640 1 Processor -- start 2024-05-27T12:35:45.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.342+0000 7f4083200640 1 -- start start 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 0x7f407c13d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 0x7f407c0abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f407c15c920 con 0x7f407c0adde0 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f407c15c5b0 con 0x7f40640039b0 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4083200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f407c071d60 con 0x7f407c0ab880 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37742/0 (socket says 172.21.15.94:37742) 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 -- 172.21.15.94:0/2029427147 learned_addr learned my addr 172.21.15.94:0/2029427147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:45.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4081e00640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 0x7f407c13d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4081400640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 0x7f407c0abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 -- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 msgr2=0x7f407c0abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 0x7f407c0abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 -- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 msgr2=0x7f407c13d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.343+0000 7f4082800640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 0x7f407c13d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4082800640 1 -- 172.21.15.94:0/2029427147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c0ac300 con 0x7f407c0adde0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4081400640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 0x7f407c0abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4081e00640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 0x7f407c13d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4082800640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f407c06d660 tx=0x7f407800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407800bb10 con 0x7f407c0adde0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407800bcb0 con 0x7f407c0adde0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780072b0 con 0x7f407c0adde0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c0ac5e0 con 0x7f407c0adde0 2024-05-27T12:35:45.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.344+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c0acba0 con 0x7f407c0adde0 2024-05-27T12:35:45.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.345+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c075460 con 0x7f407c0adde0 2024-05-27T12:35:45.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.346+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f407801b020 con 0x7f407c0adde0 2024-05-27T12:35:45.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.347+0000 7f4076000640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 0x7f405c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.347+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f407808bdc0 con 0x7f407c0adde0 2024-05-27T12:35:45.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.348+0000 7f4081e00640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 0x7f405c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.349+0000 7f4081e00640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 0x7f405c06ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4068033000 tx=0x7f4068008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.352+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f407c13e9e0 con 0x7f407c0adde0 2024-05-27T12:35:45.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.552+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f407c0cc220 con 0x7f407c0adde0 2024-05-27T12:35:45.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.555+0000 7f4076000640 1 -- 172.21.15.94:0/2029427147 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16806 (secure 0 0 0) 0x7f407800be60 con 0x7f407c0adde0 2024-05-27T12:35:45.555 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:45.555 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:45.184623+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 msgr2=0x7f405c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 0x7f405c06ee00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4068033000 tx=0x7f4068008030 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 msgr2=0x7f407c0ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f407c06d660 tx=0x7f407800df90 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 shutdown_connections 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f405c06c940 0x7f405c06ee00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f407c0adde0 0x7f407c0ae260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f407c0ab880 0x7f407c0abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 --2- 172.21.15.94:0/2029427147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40640039b0 0x7f407c13d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 >> 172.21.15.94:0/2029427147 conn(0x7f407c0b2160 msgr2=0x7f407c15d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:45.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 shutdown_connections 2024-05-27T12:35:45.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.559+0000 7f4083200640 1 -- 172.21.15.94:0/2029427147 wait complete. 2024-05-27T12:35:45.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:45.575 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:45.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.778+0000 7f6091000640 1 Processor -- start 2024-05-27T12:35:45.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.778+0000 7f6091000640 1 -- start start 2024-05-27T12:35:45.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6091000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 0x7f608c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6091000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6091000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f608c15bae0 con 0x7f608c06be70 2024-05-27T12:35:45.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6091000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f608c0cf6e0 con 0x7f608c15b700 2024-05-27T12:35:45.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6091000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f608c0b3540 con 0x7f608c06fe70 2024-05-27T12:35:45.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6089400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.780+0000 7f6089e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 0x7f608c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.781+0000 7f608a800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f608c06be70 0x7f608c06de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.94:57832/0 (socket says 172.21.15.94:57832) 2024-05-27T12:35:45.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.781+0000 7f6089400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:37994/0 (socket says 172.21.15.94:37994) 2024-05-27T12:35:45.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.781+0000 7f608a800640 1 -- 172.21.15.94:0/385246887 learned_addr learned my addr 172.21.15.94:0/385246887 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:45.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.781+0000 7f6089e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 0x7f608c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:43818/0 (socket says 172.21.15.94:43818) 2024-05-27T12:35:45.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.782+0000 7f6089400640 1 -- 172.21.15.94:0/385246887 >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 msgr2=0x7f608c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:45.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.782+0000 7f6089400640 1 --2- 172.21.15.94:0/385246887 >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 0x7f608c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.782+0000 7f6089400640 1 -- 172.21.15.94:0/385246887 >> v1:172.21.15.97:6789/0 conn(0x7f608c06be70 legacy=0x7f608c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.782+0000 7f6089400640 1 -- 172.21.15.94:0/385246887 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c0b3820 con 0x7f608c06fe70 2024-05-27T12:35:45.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.783+0000 7f6089400640 1 --2- 172.21.15.94:0/385246887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60780097a0 tx=0x7f607802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad4f138b8fe42a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.784+0000 7f6088a00640 1 -- 172.21.15.94:0/385246887 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078037070 con 0x7f608c06fe70 2024-05-27T12:35:45.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.784+0000 7f6088a00640 1 -- 172.21.15.94:0/385246887 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607802ae50 con 0x7f608c06fe70 2024-05-27T12:35:45.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.784+0000 7f6088a00640 1 -- 172.21.15.94:0/385246887 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078032660 con 0x7f608c06fe70 2024-05-27T12:35:45.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.784+0000 7f6091000640 1 -- 172.21.15.94:0/385246887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 msgr2=0x7f608c06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.784+0000 7f6091000640 1 --2- 172.21.15.94:0/385246887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60780097a0 tx=0x7f607802a100 comp rx=0 tx=0).stop 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 -- 172.21.15.94:0/385246887 shutdown_connections 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 --2- 172.21.15.94:0/385246887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c06fe70 0x7f608c06b930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 --2- 172.21.15.94:0/385246887 >> v2:172.21.15.97:3300/0 conn(0x7f608c15b700 0x7f608c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 -- 172.21.15.94:0/385246887 >> 172.21.15.94:0/385246887 conn(0x7f608c0b2240 msgr2=0x7f608c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 -- 172.21.15.94:0/385246887 shutdown_connections 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.785+0000 7f6091000640 1 -- 172.21.15.94:0/385246887 wait complete. 2024-05-27T12:35:45.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.786+0000 7f6091000640 1 Processor -- start 2024-05-27T12:35:45.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.786+0000 7f6091000640 1 -- start start 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 0x7f608c13eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 0x7f608c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f608c0b3de0 con 0x7f608c06fe70 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f608c06c280 con 0x7f608c06be70 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.787+0000 7f6091000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f608c0cf3a0 con 0x7f608c15b700 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37758/0 (socket says 172.21.15.94:37758) 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 -- 172.21.15.94:0/3527535292 learned_addr learned my addr 172.21.15.94:0/3527535292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f608a800640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 0x7f608c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089e00640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 0x7f608c13eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 -- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 msgr2=0x7f608c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 0x7f608c150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 -- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 msgr2=0x7f608c13eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 0x7f608c13eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089400640 1 -- 172.21.15.94:0/3527535292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c1f44e0 con 0x7f608c06fe70 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f608a800640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 0x7f608c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.788+0000 7f6089e00640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 0x7f608c13eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6089400640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60780097a0 tx=0x7f60780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078042070 con 0x7f608c06fe70 2024-05-27T12:35:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607803e460 con 0x7f608c06fe70 2024-05-27T12:35:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078037040 con 0x7f608c06fe70 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c153cb0 con 0x7f608c06fe70 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.789+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c14ebd0 con 0x7f608c06fe70 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.790+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608c068a70 con 0x7f608c06fe70 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.791+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6078045070 con 0x7f608c06fe70 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.792+0000 7f6082000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.792+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f60780ac300 con 0x7f608c06fe70 2024-05-27T12:35:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.793+0000 7f6089e00640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.794+0000 7f6089e00640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 0x7f606c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b4470 tx=0x7f607c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:45.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.797+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f607807a730 con 0x7f608c06fe70 2024-05-27T12:35:45.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.998+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f608c06c5e0 con 0x7f608c06fe70 2024-05-27T12:35:45.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:45.998+0000 7f6082000640 1 -- 172.21.15.94:0/3527535292 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f607807a030 con 0x7f608c06fe70 2024-05-27T12:35:45.998 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.001+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 msgr2=0x7f606c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.001+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 0x7f606c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b4470 tx=0x7f607c008040 comp rx=0 tx=0).stop 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.001+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 msgr2=0x7f608c15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.001+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60780097a0 tx=0x7f60780046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 shutdown_connections 2024-05-27T12:35:46.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f606c06c8f0 0x7f606c06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f608c15b700 0x7f608c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f608c06fe70 0x7f608c15e490 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 --2- 172.21.15.94:0/3527535292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f608c06be70 0x7f608c13eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 >> 172.21.15.94:0/3527535292 conn(0x7f608c0b2240 msgr2=0x7f608c0d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 shutdown_connections 2024-05-27T12:35:46.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.002+0000 7f6091000640 1 -- 172.21.15.94:0/3527535292 wait complete. 2024-05-27T12:35:46.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:46.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.227+0000 7f951ae00640 1 Processor -- start 2024-05-27T12:35:46.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.228+0000 7f951ae00640 1 -- start start 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951ae00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951ae00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f95140ce770 0x7f95140d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951ae00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95140cf570 con 0x7f95140d1910 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951ae00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f95140d0dd0 con 0x7f95140ce770 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951ae00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f951415ba50 con 0x7f951415b670 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951a400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f95140d1910 0x7f95140c6690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51378/0 (socket says 172.21.15.94:51378) 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f951a400640 1 -- 172.21.15.94:0/742237148 learned_addr learned my addr 172.21.15.94:0/742237148 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:46.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f95140ce770 0x7f95140d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519a00640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519000640 1 -- 172.21.15.94:0/742237148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 msgr2=0x7f95140d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519000640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519000640 1 -- 172.21.15.94:0/742237148 >> v1:172.21.15.94:6789/0 conn(0x7f95140d1910 legacy=0x7f95140c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519000640 1 -- 172.21.15.94:0/742237148 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95140cf7f0 con 0x7f95140ce770 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9519a00640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.230+0000 7f9513e00640 1 -- 172.21.15.94:0/742237148 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926530513 0 0) 0x7f950c001110 con 0x7f95140d1910 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.231+0000 7f9519000640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140d13d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9508009d90 tx=0x7f950802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35666a06d803efb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.231+0000 7f9513e00640 1 -- 172.21.15.94:0/742237148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508007740 con 0x7f95140ce770 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.231+0000 7f9513e00640 1 -- 172.21.15.94:0/742237148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95080078e0 con 0x7f95140ce770 2024-05-27T12:35:46.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.231+0000 7f9513e00640 1 -- 172.21.15.94:0/742237148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508007cb0 con 0x7f95140ce770 2024-05-27T12:35:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 -- 172.21.15.94:0/742237148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 msgr2=0x7f95140d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140d13d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9508009d90 tx=0x7f950802a660 comp rx=0 tx=0).stop 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 -- 172.21.15.94:0/742237148 shutdown_connections 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140d13d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 --2- 172.21.15.94:0/742237148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 -- 172.21.15.94:0/742237148 >> 172.21.15.94:0/742237148 conn(0x7f95140734b0 msgr2=0x7f951409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 -- 172.21.15.94:0/742237148 shutdown_connections 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.233+0000 7f951ae00640 1 -- 172.21.15.94:0/742237148 wait complete. 2024-05-27T12:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.234+0000 7f951ae00640 1 Processor -- start 2024-05-27T12:35:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.234+0000 7f951ae00640 1 -- start start 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95140d1910 0x7f95141500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95140d0090 con 0x7f95140ce770 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95140cfdb0 con 0x7f95140d1910 2024-05-27T12:35:46.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f951ae00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f95140d0370 con 0x7f951415b670 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f9519a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f9519a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37772/0 (socket says 172.21.15.94:37772) 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.235+0000 7f9519a00640 1 -- 172.21.15.94:0/4120319540 learned_addr learned my addr 172.21.15.94:0/4120319540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f951a400640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519000640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95140d1910 0x7f95141500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 -- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 msgr2=0x7f95140b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 -- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95140d1910 msgr2=0x7f95141500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95140d1910 0x7f95141500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 -- 172.21.15.94:0/4120319540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95140b19d0 con 0x7f95140ce770 2024-05-27T12:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f951a400640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9519a00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f950400c7f0 tx=0x7f950400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.236+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504015070 con 0x7f95140ce770 2024-05-27T12:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.238+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95040079d0 con 0x7f95140ce770 2024-05-27T12:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.238+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504007da0 con 0x7f95140ce770 2024-05-27T12:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.238+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9514150620 con 0x7f95140ce770 2024-05-27T12:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.238+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95140b5470 con 0x7f95140ce770 2024-05-27T12:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.239+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9514075460 con 0x7f95140ce770 2024-05-27T12:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.240+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9504018070 con 0x7f95140ce770 2024-05-27T12:35:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.241+0000 7f9511600640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.241+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f950408bb90 con 0x7f95140ce770 2024-05-27T12:35:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.242+0000 7f9519000640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.243+0000 7f9519000640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 0x7f94f806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f950802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.244+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9514150620 con 0x7f95140ce770 2024-05-27T12:35:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.447+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95140b7710 con 0x7f95140ce770 2024-05-27T12:35:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.447+0000 7f9511600640 1 -- 172.21.15.94:0/4120319540 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f9504059ea0 con 0x7f95140ce770 2024-05-27T12:35:46.446 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 msgr2=0x7f94f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 0x7f94f806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f950802f000 comp rx=0 tx=0).stop 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 msgr2=0x7f95140b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f950400c7f0 tx=0x7f950400cbf0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 shutdown_connections 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f94f806c8f0 0x7f94f806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95140d1910 0x7f95141500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f951415b670 0x7f95140b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 --2- 172.21.15.94:0/4120319540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95140ce770 0x7f95140b4e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 >> 172.21.15.94:0/4120319540 conn(0x7f95140734b0 msgr2=0x7f95140afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.450+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 shutdown_connections 2024-05-27T12:35:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.451+0000 7f951ae00640 1 -- 172.21.15.94:0/4120319540 wait complete. 2024-05-27T12:35:46.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.667+0000 7f7a71800640 1 Processor -- start 2024-05-27T12:35:46.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.668+0000 7f7a71800640 1 -- start start 2024-05-27T12:35:46.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a71800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a6c13f480 0x7f7a6c071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a71800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c15d2f0 0x7f7a6c140a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a71800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a6c06dc50 con 0x7f7a6c072260 2024-05-27T12:35:46.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a71800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7a6c13fcd0 con 0x7f7a6c13f480 2024-05-27T12:35:46.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a71800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6c140570 con 0x7f7a6c15d2f0 2024-05-27T12:35:46.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a63e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7a6c072260 0x7f7a6c15cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51394/0 (socket says 172.21.15.94:51394) 2024-05-27T12:35:46.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a63e00640 1 -- 172.21.15.94:0/3488562820 learned_addr learned my addr 172.21.15.94:0/3488562820 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a6be00640 1 --2- 172.21.15.94:0/3488562820 >> v2:172.21.15.97:3300/0 conn(0x7f7a6c13f480 0x7f7a6c071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a70e00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c15d2f0 0x7f7a6c140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605913943 0 0) 0x7f7a54001110 con 0x7f7a6c072260 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.670+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a50003520 con 0x7f7a6c072260 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1318210192 0 0) 0x7f7a50003520 con 0x7f7a6c072260 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c15d2f0 msgr2=0x7f7a6c140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c15d2f0 0x7f7a6c140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 >> v2:172.21.15.97:3300/0 conn(0x7f7a6c13f480 msgr2=0x7f7a6c071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 --2- 172.21.15.94:0/3488562820 >> v2:172.21.15.97:3300/0 conn(0x7f7a6c13f480 0x7f7a6c071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c15d950 con 0x7f7a6c072260 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f7a54001840 con 0x7f7a6c072260 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 >> v1:172.21.15.94:6789/0 conn(0x7f7a6c072260 legacy=0x7f7a6c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a50004800 0x7f7a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a500051c0 0x7f7a5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c13fcd0 con 0x7f7a500039b0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c140570 con 0x7f7a500051c0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5000c000 con 0x7f7a50004800 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a70e00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c15d2f0 0x7f7a6c140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.671+0000 7f7a6be00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a63e00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a50004800 0x7f7a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a6be00640 1 -- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a50004800 msgr2=0x7f7a50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a6be00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a50004800 0x7f7a50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a70e00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a500051c0 0x7f7a5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a6be00640 1 -- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a500051c0 msgr2=0x7f7a5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a6be00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a500051c0 0x7f7a5000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a6be00640 1 -- 172.21.15.94:0/3488562820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a50003520 con 0x7f7a500039b0 2024-05-27T12:35:46.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.672+0000 7f7a63e00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a50004800 0x7f7a50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.673+0000 7f7a6be00640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00ba60 tx=0x7f7a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc25f871911f9943 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.673+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c030070 con 0x7f7a500039b0 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.673+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c02ae40 con 0x7f7a500039b0 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.673+0000 7f7a6b400640 1 -- 172.21.15.94:0/3488562820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00b600 con 0x7f7a500039b0 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 -- 172.21.15.94:0/3488562820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 msgr2=0x7f7a50003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 --2- 172.21.15.94:0/3488562820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a50003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00ba60 tx=0x7f7a50005830 comp rx=0 tx=0).stop 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a70e00640 1 -- 172.21.15.94:0/3488562820 reap_dead start 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 -- 172.21.15.94:0/3488562820 shutdown_connections 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 -- 172.21.15.94:0/3488562820 >> 172.21.15.94:0/3488562820 conn(0x7f7a6c0b6230 msgr2=0x7f7a6c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 -- 172.21.15.94:0/3488562820 shutdown_connections 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.674+0000 7f7a71800640 1 -- 172.21.15.94:0/3488562820 wait complete. 2024-05-27T12:35:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.676+0000 7f7a71800640 1 Processor -- start 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.676+0000 7f7a71800640 1 -- start start 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6c1f79c0 0x7f7a6c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c160080 0x7f7a6c160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c13f860 con 0x7f7a500039b0 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c140200 con 0x7f7a6c1f79c0 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a71800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6c13f9e0 con 0x7f7a6c160080 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37788/0 (socket says 172.21.15.94:37788) 2024-05-27T12:35:46.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 -- 172.21.15.94:0/675333047 learned_addr learned my addr 172.21.15.94:0/675333047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 -- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c160080 msgr2=0x7f7a6c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c160080 0x7f7a6c160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 -- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6c1f79c0 msgr2=0x7f7a6c0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6c1f79c0 0x7f7a6c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.677+0000 7f7a6be00640 1 -- 172.21.15.94:0/675333047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c159f90 con 0x7f7a500039b0 2024-05-27T12:35:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a6be00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00bb50 tx=0x7f7a5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c042020 con 0x7f7a500039b0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c03d650 con 0x7f7a500039b0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c03da40 con 0x7f7a500039b0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c0b51f0 con 0x7f7a500039b0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.678+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1591d0 con 0x7f7a500039b0 2024-05-27T12:35:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.679+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c160a40 con 0x7f7a500039b0 2024-05-27T12:35:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.680+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a5c045020 con 0x7f7a500039b0 2024-05-27T12:35:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.681+0000 7f7a68c00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 0x7f7a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.681+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7a5c0ac330 con 0x7f7a500039b0 2024-05-27T12:35:46.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.682+0000 7f7a63e00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 0x7f7a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.683+0000 7f7a63e00640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 0x7f7a4806ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a540012f0 tx=0x7f7a54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:46.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.686+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a5c04c030 con 0x7f7a500039b0 2024-05-27T12:35:46.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.888+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a6c075460 con 0x7f7a500039b0 2024-05-27T12:35:46.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.889+0000 7f7a68c00640 1 -- 172.21.15.94:0/675333047 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f7a5c07a6e0 con 0x7f7a500039b0 2024-05-27T12:35:46.887 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:46.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 msgr2=0x7f7a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 0x7f7a4806ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a540012f0 tx=0x7f7a54009040 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 msgr2=0x7f7a6c158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00bb50 tx=0x7f7a5c004730 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 shutdown_connections 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7a4806c920 0x7f7a4806ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6c1f79c0 0x7f7a6c0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a6c160080 0x7f7a6c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 --2- 172.21.15.94:0/675333047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a500039b0 0x7f7a6c158880 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 >> 172.21.15.94:0/675333047 conn(0x7f7a6c0b6230 msgr2=0x7f7a6c15e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 shutdown_connections 2024-05-27T12:35:46.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:46.892+0000 7f7a71800640 1 -- 172.21.15.94:0/675333047 wait complete. 2024-05-27T12:35:46.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T12:35:47.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.115+0000 7f2d64e00640 1 Processor -- start 2024-05-27T12:35:47.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.116+0000 7f2d64e00640 1 -- start start 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.117+0000 7f2d64e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d60071d40 0x7f2d600721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.117+0000 7f2d64e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d60161000 con 0x7f2d60160140 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.117+0000 7f2d64e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d60160d20 con 0x7f2d60145df0 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.117+0000 7f2d64e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2d600726f0 con 0x7f2d60071d40 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2d60160140 0x7f2d60160570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51396/0 (socket says 172.21.15.94:51396) 2024-05-27T12:35:47.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5fe00640 1 -- 172.21.15.94:0/2820380564 learned_addr learned my addr 172.21.15.94:0/2820380564 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/2820380564 >> v2:172.21.15.97:3300/0 conn(0x7f2d60071d40 0x7f2d600721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504221665 0 0) 0x7f2d60161000 con 0x7f2d60160140 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d40003520 con 0x7f2d60160140 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.118+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3593020424 0 0) 0x7f2d40003520 con 0x7f2d60160140 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 >> v2:172.21.15.97:3300/0 conn(0x7f2d60071d40 msgr2=0x7f2d600721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 --2- 172.21.15.94:0/2820380564 >> v2:172.21.15.97:3300/0 conn(0x7f2d60071d40 0x7f2d600721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 >> v1:172.21.15.97:6789/0 conn(0x7f2d60145df0 legacy=0x7f2d600cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d600ccf30 con 0x7f2d60160140 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f2d580019f0 con 0x7f2d60160140 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 >> v1:172.21.15.94:6789/0 conn(0x7f2d60160140 legacy=0x7f2d60160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400039b0 0x7f2d40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 0x7f2d40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d400051c0 0x7f2d4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600726f0 con 0x7f2d40004800 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4000c160 con 0x7f2d400051c0 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4000c360 con 0x7f2d400039b0 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 0x7f2d40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 -- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400039b0 msgr2=0x7f2d40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400039b0 0x7f2d40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 -- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d400051c0 msgr2=0x7f2d4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:47.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d400051c0 0x7f2d4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5ea00640 1 -- 172.21.15.94:0/2820380564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d40003520 con 0x7f2d40004800 2024-05-27T12:35:47.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.119+0000 7f2d5fe00640 1 -- 172.21.15.94:0/2820380564 reap_dead start 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.120+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 0x7f2d40004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009ac0 tx=0x7f2d40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e4dc6398f250e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.120+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48036070 con 0x7f2d40004800 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48031400 con 0x7f2d40004800 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d5e000640 1 -- 172.21.15.94:0/2820380564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48009690 con 0x7f2d40004800 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 -- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 msgr2=0x7f2d40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 0x7f2d40004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009ac0 tx=0x7f2d40005830 comp rx=0 tx=0).stop 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 -- 172.21.15.94:0/2820380564 shutdown_connections 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 --2- 172.21.15.94:0/2820380564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d40004800 0x7f2d40004c80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 -- 172.21.15.94:0/2820380564 >> 172.21.15.94:0/2820380564 conn(0x7f2d6006b930 msgr2=0x7f2d6006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.121+0000 7f2d64e00640 1 -- 172.21.15.94:0/2820380564 shutdown_connections 2024-05-27T12:35:47.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.122+0000 7f2d64e00640 1 -- 172.21.15.94:0/2820380564 wait complete. 2024-05-27T12:35:47.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.123+0000 7f2d64e00640 1 Processor -- start 2024-05-27T12:35:47.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.123+0000 7f2d64e00640 1 -- start start 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d40004800 0x7f2d60142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d600b1350 0x7f2d6015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601461d0 con 0x7f2d601f25f0 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600cd4f0 con 0x7f2d40004800 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.124+0000 7f2d64e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2d600721b0 con 0x7f2d600b1350 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37812/0 (socket says 172.21.15.94:37812) 2024-05-27T12:35:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 -- 172.21.15.94:0/1040654002 learned_addr learned my addr 172.21.15.94:0/1040654002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5fe00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d600b1350 0x7f2d6015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5f400640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d40004800 0x7f2d60142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 -- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d600b1350 msgr2=0x7f2d6015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d600b1350 0x7f2d6015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 -- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d40004800 msgr2=0x7f2d60142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d40004800 0x7f2d60142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.125+0000 7f2d5ea00640 1 -- 172.21.15.94:0/1040654002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d601f2b60 con 0x7f2d601f25f0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d5ea00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802abb0 tx=0x7f2d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4803f020 con 0x7f2d601f25f0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48037040 con 0x7f2d601f25f0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d601f2e40 con 0x7f2d601f25f0 2024-05-27T12:35:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48036070 con 0x7f2d601f25f0 2024-05-27T12:35:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.126+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d601f3400 con 0x7f2d601f25f0 2024-05-27T12:35:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.127+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d60075460 con 0x7f2d601f25f0 2024-05-27T12:35:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.128+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d48042020 con 0x7f2d601f25f0 2024-05-27T12:35:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.129+0000 7f2d57400640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.129+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2d480ac080 con 0x7f2d601f25f0 2024-05-27T12:35:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.130+0000 7f2d5f400640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.131+0000 7f2d5f400640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d600690b0 tx=0x7f2d50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.133+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d48049020 con 0x7f2d601f25f0 2024-05-27T12:35:47.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.344+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d601609c0 con 0x7f2d601f25f0 2024-05-27T12:35:47.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.344+0000 7f2d57400640 1 -- 172.21.15.94:0/1040654002 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f2d4807a450 con 0x7f2d601f25f0 2024-05-27T12:35:47.343 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-05-27T12:35:47.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 msgr2=0x7f2d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d600690b0 tx=0x7f2d50009040 comp rx=0 tx=0).stop 2024-05-27T12:35:47.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 msgr2=0x7f2d6013bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d4802abb0 tx=0x7f2d48004730 comp rx=0 tx=0).stop 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 shutdown_connections 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2d3c06c8f0 0x7f2d3c06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d600b1350 0x7f2d6015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d601f25f0 0x7f2d6013bd90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 --2- 172.21.15.94:0/1040654002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d40004800 0x7f2d60142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 >> 172.21.15.94:0/1040654002 conn(0x7f2d6006b930 msgr2=0x7f2d600cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 shutdown_connections 2024-05-27T12:35:47.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.347+0000 7f2d64e00640 1 -- 172.21.15.94:0/1040654002 wait complete. 2024-05-27T12:35:47.363 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-27T12:35:47.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:47.363 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-27T12:35:47.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-05-27T12:35:47.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:47.442 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-27T12:35:47.491 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:47.491 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:47.550 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:47.550 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:47.602 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-05-27T12:35:47.603 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:35:47.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:47.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.862+0000 7ff80dc00640 1 Processor -- start 2024-05-27T12:35:47.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.862+0000 7ff80dc00640 1 -- start start 2024-05-27T12:35:47.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.863+0000 7ff80dc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff80815a250 0x7ff80806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.863+0000 7ff80dc00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff80806f280 0x7ff8080cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.863+0000 7ff80dc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff808160360 con 0x7ff8080cf940 2024-05-27T12:35:47.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.863+0000 7ff80dc00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff8080cfdb0 con 0x7ff80806f280 2024-05-27T12:35:47.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.863+0000 7ff80dc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff808160f70 con 0x7ff80815a250 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff80806f280 0x7ff8080cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff80806f280 0x7ff8080cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37814/0 (socket says 172.21.15.94:37814) 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 -- 172.21.15.94:0/3814323112 learned_addr learned my addr 172.21.15.94:0/3814323112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806a00640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff80815a250 0x7ff80806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 -- 172.21.15.94:0/3814323112 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff80815a250 msgr2=0x7ff80806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff80815a250 0x7ff80806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 -- 172.21.15.94:0/3814323112 >> v1:172.21.15.97:6789/0 conn(0x7ff8080cf940 legacy=0x7ff808071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.864+0000 7ff806000640 1 -- 172.21.15.94:0/3814323112 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff808160640 con 0x7ff80806f280 2024-05-27T12:35:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.865+0000 7ff806000640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80806f280 0x7ff8080cf400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009870 tx=0x7ff7f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8059a206f7fe86f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff805600640 1 -- 172.21.15.94:0/3814323112 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4037070 con 0x7ff80806f280 2024-05-27T12:35:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff805600640 1 -- 172.21.15.94:0/3814323112 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f4032440 con 0x7ff80806f280 2024-05-27T12:35:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff805600640 1 -- 172.21.15.94:0/3814323112 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4032810 con 0x7ff80806f280 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 -- 172.21.15.94:0/3814323112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80806f280 msgr2=0x7ff8080cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80806f280 0x7ff8080cf400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009870 tx=0x7ff7f402a210 comp rx=0 tx=0).stop 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 -- 172.21.15.94:0/3814323112 shutdown_connections 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80806f280 0x7ff8080cf400 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3814323112 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff80815a250 0x7ff80806ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 -- 172.21.15.94:0/3814323112 >> 172.21.15.94:0/3814323112 conn(0x7ff8080b3830 msgr2=0x7ff8080b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 -- 172.21.15.94:0/3814323112 shutdown_connections 2024-05-27T12:35:47.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.866+0000 7ff80dc00640 1 -- 172.21.15.94:0/3814323112 wait complete. 2024-05-27T12:35:47.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.868+0000 7ff80dc00640 1 Processor -- start 2024-05-27T12:35:47.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.868+0000 7ff80dc00640 1 -- start start 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff80806f280 0x7ff80815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8080cf940 0x7ff80814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff80815a630 con 0x7ff80815a250 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff808160c00 con 0x7ff80806f280 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff80dc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff80806e7f0 con 0x7ff8080cf940 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.869+0000 7ff807400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37816/0 (socket says 172.21.15.94:37816) 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 -- 172.21.15.94:0/3016341800 learned_addr learned my addr 172.21.15.94:0/3016341800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff806000640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8080cf940 0x7ff80814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff806a00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff80806f280 0x7ff80815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 -- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8080cf940 msgr2=0x7ff80814f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8080cf940 0x7ff80814f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 -- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff80806f280 msgr2=0x7ff80815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff80806f280 0x7ff80815bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 -- 172.21.15.94:0/3016341800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80815ac20 con 0x7ff80815a250 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff807400640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c780 tx=0x7ff80000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800004b60 con 0x7ff80815a250 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff800004d00 con 0x7ff80815a250 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.870+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80815af00 con 0x7ff80815a250 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.871+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80000aa90 con 0x7ff80815a250 2024-05-27T12:35:47.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.871+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80815b4c0 con 0x7ff80815a250 2024-05-27T12:35:47.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.872+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff808075460 con 0x7ff80815a250 2024-05-27T12:35:47.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.873+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff80000ac30 con 0x7ff80815a250 2024-05-27T12:35:47.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.874+0000 7ff7fea00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 0x7ff7e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:47.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.874+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff80008bf80 con 0x7ff80815a250 2024-05-27T12:35:47.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.874+0000 7ff806a00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 0x7ff7e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:47.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.876+0000 7ff806a00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 0x7ff7e806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006fd0 tx=0x7ff7f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:47.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:47.878+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff800020070 con 0x7ff80815a250 2024-05-27T12:35:48.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.086+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8080c7eb0 con 0x7ff80815a250 2024-05-27T12:35:48.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.089+0000 7ff7fea00640 1 -- 172.21.15.94:0/3016341800 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7ff80005a290 con 0x7ff80815a250 2024-05-27T12:35:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:48.088 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:46.194696+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 msgr2=0x7ff7e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 0x7ff7e806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006fd0 tx=0x7ff7f8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 msgr2=0x7ff80814f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c780 tx=0x7ff80000cc40 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 shutdown_connections 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff7e806c8f0 0x7ff7e806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80815a250 0x7ff80814f7b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8080cf940 0x7ff80814f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 --2- 172.21.15.94:0/3016341800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff80806f280 0x7ff80815bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 >> 172.21.15.94:0/3016341800 conn(0x7ff8080b3830 msgr2=0x7ff8080b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 shutdown_connections 2024-05-27T12:35:48.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.091+0000 7ff80dc00640 1 -- 172.21.15.94:0/3016341800 wait complete. 2024-05-27T12:35:48.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:48.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.310+0000 7f66fb400640 1 Processor -- start 2024-05-27T12:35:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.310+0000 7f66fb400640 1 -- start start 2024-05-27T12:35:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66fb400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f66f406ca40 0x7f66f409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66fb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66fb400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f40d2020 con 0x7f66f4162150 2024-05-27T12:35:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66fb400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f66f409b3a0 con 0x7f66f406ca40 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66fb400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40d1d40 con 0x7f66f409af70 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66f3e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f66f406ca40 0x7f66f409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66f3e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f66f406ca40 0x7f66f409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37818/0 (socket says 172.21.15.94:37818) 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.312+0000 7f66f3e00640 1 -- 172.21.15.94:0/861191012 learned_addr learned my addr 172.21.15.94:0/861191012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.313+0000 7f66f3e00640 1 -- 172.21.15.94:0/861191012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 msgr2=0x7f66f40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.313+0000 7f66f9400640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.313+0000 7f66f3e00640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.313+0000 7f66f3e00640 1 -- 172.21.15.94:0/861191012 >> v1:172.21.15.97:6789/0 conn(0x7f66f4162150 legacy=0x7f66f40d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.313+0000 7f66f3e00640 1 -- 172.21.15.94:0/861191012 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f40ccd60 con 0x7f66f406ca40 2024-05-27T12:35:48.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.314+0000 7f66f3e00640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f409aa30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009870 tx=0x7f66e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ee775c3605fc6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.314+0000 7f66f3400640 1 -- 172.21.15.94:0/861191012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0037070 con 0x7f66f406ca40 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.314+0000 7f66f3400640 1 -- 172.21.15.94:0/861191012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0032440 con 0x7f66f406ca40 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.314+0000 7f66f3400640 1 -- 172.21.15.94:0/861191012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0032810 con 0x7f66f406ca40 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 -- 172.21.15.94:0/861191012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 msgr2=0x7f66f409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f409aa30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009870 tx=0x7f66e002a210 comp rx=0 tx=0).stop 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 -- 172.21.15.94:0/861191012 shutdown_connections 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 --2- 172.21.15.94:0/861191012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f409aa30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 -- 172.21.15.94:0/861191012 >> 172.21.15.94:0/861191012 conn(0x7f66f40b4160 msgr2=0x7f66f40b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 -- 172.21.15.94:0/861191012 shutdown_connections 2024-05-27T12:35:48.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.315+0000 7f66fb400640 1 -- 172.21.15.94:0/861191012 wait complete. 2024-05-27T12:35:48.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.317+0000 7f66fb400640 1 Processor -- start 2024-05-27T12:35:48.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.317+0000 7f66fb400640 1 -- start start 2024-05-27T12:35:48.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f4162150 0x7f66f40c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40cdd80 con 0x7f66f406ca40 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40cd320 con 0x7f66f4162150 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66fb400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40cdf00 con 0x7f66f409af70 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66f8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66f8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37824/0 (socket says 172.21.15.94:37824) 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.318+0000 7f66f8a00640 1 -- 172.21.15.94:0/1468007126 learned_addr learned my addr 172.21.15.94:0/1468007126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 -- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 msgr2=0x7f66f40b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 -- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f4162150 msgr2=0x7f66f40c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f4162150 0x7f66f40c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 -- 172.21.15.94:0/1468007126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f4159d00 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f8a00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3f0 tx=0x7f66e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4015020 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e4011830 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011c20 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f40c66d0 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.319+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f40c6c90 con 0x7f66f406ca40 2024-05-27T12:35:48.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.320+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f40b5a80 con 0x7f66f406ca40 2024-05-27T12:35:48.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.321+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66e4018020 con 0x7f66f406ca40 2024-05-27T12:35:48.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.322+0000 7f66f0c00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.322+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f66e408b810 con 0x7f66f406ca40 2024-05-27T12:35:48.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.323+0000 7f66f3e00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.324+0000 7f66f3e00640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 0x7f66d406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.326+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66f40b5a80 con 0x7f66f406ca40 2024-05-27T12:35:48.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.523+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66f40757c0 con 0x7f66f406ca40 2024-05-27T12:35:48.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.523+0000 7f66f0c00640 1 -- 172.21.15.94:0/1468007126 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f66e401f030 con 0x7f66f406ca40 2024-05-27T12:35:48.522 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 msgr2=0x7f66d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 0x7f66d406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66e0034000 tx=0x7f66e002c000 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 msgr2=0x7f66f41597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3f0 tx=0x7f66e400b8b0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 shutdown_connections 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f66d406c8f0 0x7f66d406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f4162150 0x7f66f40c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66f409af70 0x7f66f40b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 --2- 172.21.15.94:0/1468007126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f406ca40 0x7f66f41597c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 >> 172.21.15.94:0/1468007126 conn(0x7f66f40b4160 msgr2=0x7f66f40ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:48.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.526+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 shutdown_connections 2024-05-27T12:35:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.527+0000 7f66fb400640 1 -- 172.21.15.94:0/1468007126 wait complete. 2024-05-27T12:35:48.544 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.753+0000 7f314e800640 1 Processor -- start 2024-05-27T12:35:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.753+0000 7f314e800640 1 -- start start 2024-05-27T12:35:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314e800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f3148067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314e800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31480b7840 con 0x7f31480ba3f0 2024-05-27T12:35:48.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314e800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3148067f30 con 0x7f31480b68a0 2024-05-27T12:35:48.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314e800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3148068210 con 0x7f314815e900 2024-05-27T12:35:48.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314ca00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f31480ba3f0 0x7f31480b6190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51408/0 (socket says 172.21.15.94:51408) 2024-05-27T12:35:48.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.755+0000 7f314ca00640 1 -- 172.21.15.94:0/2178715733 learned_addr learned my addr 172.21.15.94:0/2178715733 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f314d400640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f3148067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206913240 0 0) 0x7f3134001490 con 0x7f31480ba3f0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f312c003520 con 0x7f31480ba3f0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1855662218 0 0) 0x7f312c003520 con 0x7f31480ba3f0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 msgr2=0x7f3148067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f3148067630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 >> v1:172.21.15.97:6789/0 conn(0x7f31480b68a0 legacy=0x7f31480d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31480684f0 con 0x7f31480ba3f0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f3134001bc0 con 0x7f31480ba3f0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 >> v1:172.21.15.94:6789/0 conn(0x7f31480ba3f0 legacy=0x7f31480b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.756+0000 7f3147e00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c0039e0 0x7f312c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f3147e00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f3147e00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f312c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3148068210 con 0x7f312c0039e0 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f312c00bb00 con 0x7f312c004800 2024-05-27T12:35:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f312c00bd00 con 0x7f314815e900 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c0039e0 0x7f312c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314ca00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314de00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f312c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 -- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 msgr2=0x7f312c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f314815e900 0x7f312c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 -- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c004800 msgr2=0x7f312c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314d400640 1 -- 172.21.15.94:0/2178715733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c003520 con 0x7f312c0039e0 2024-05-27T12:35:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.757+0000 7f314ca00640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.758+0000 7f314d400640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c0039e0 0x7f312c003e00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3138009a90 tx=0x7f312c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e62a4c7a30ce96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138036070 con 0x7f312c0039e0 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138031400 con 0x7f312c0039e0 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f3147e00640 1 -- 172.21.15.94:0/2178715733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138009690 con 0x7f312c0039e0 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 -- 172.21.15.94:0/2178715733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c0039e0 msgr2=0x7f312c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 --2- 172.21.15.94:0/2178715733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c0039e0 0x7f312c003e00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3138009a90 tx=0x7f312c005830 comp rx=0 tx=0).stop 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314de00640 1 -- 172.21.15.94:0/2178715733 reap_dead start 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 -- 172.21.15.94:0/2178715733 shutdown_connections 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 -- 172.21.15.94:0/2178715733 >> 172.21.15.94:0/2178715733 conn(0x7f31480d1190 msgr2=0x7f31480d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 -- 172.21.15.94:0/2178715733 shutdown_connections 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.759+0000 7f314e800640 1 -- 172.21.15.94:0/2178715733 wait complete. 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.761+0000 7f314e800640 1 Processor -- start 2024-05-27T12:35:48.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.761+0000 7f314e800640 1 -- start start 2024-05-27T12:35:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039e0 0x7f314815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3148157a40 0x7f31480c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f314815ece0 con 0x7f3148154bf0 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31480d2df0 con 0x7f312c0039e0 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.762+0000 7f314e800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3148067630 con 0x7f3148157a40 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37836/0 (socket says 172.21.15.94:37836) 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 -- 172.21.15.94:0/3922117922 learned_addr learned my addr 172.21.15.94:0/3922117922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314d400640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039e0 0x7f314815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314ca00640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3148157a40 0x7f31480c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 -- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3148157a40 msgr2=0x7f31480c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3148157a40 0x7f31480c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 -- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039e0 msgr2=0x7f314815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039e0 0x7f314815ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 -- 172.21.15.94:0/3922117922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148160690 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f314de00640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f31480cf140 tx=0x7f314000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.763+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140018070 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.764+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31400035f0 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.764+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140003a20 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.764+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3148160970 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.764+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3148160f30 con 0x7f3148154bf0 2024-05-27T12:35:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.765+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3148075460 con 0x7f3148154bf0 2024-05-27T12:35:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.766+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f314001b070 con 0x7f3148154bf0 2024-05-27T12:35:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.767+0000 7f3145600640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 0x7f312806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.767+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f314008b1a0 con 0x7f3148154bf0 2024-05-27T12:35:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.768+0000 7f314d400640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 0x7f312806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:48.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.769+0000 7f314d400640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 0x7f312806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3138033000 tx=0x7f3138008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:48.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.771+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31400594b0 con 0x7f3148154bf0 2024-05-27T12:35:48.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.973+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3148158380 con 0x7f3148154bf0 2024-05-27T12:35:48.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.974+0000 7f3145600640 1 -- 172.21.15.94:0/3922117922 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3140058db0 con 0x7f3148154bf0 2024-05-27T12:35:48.972 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 msgr2=0x7f312806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 0x7f312806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3138033000 tx=0x7f3138008030 comp rx=0 tx=0).stop 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 msgr2=0x7f3148155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f31480cf140 tx=0x7f314000d890 comp rx=0 tx=0).stop 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 shutdown_connections 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f312806c8f0 0x7f312806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3148154bf0 0x7f3148155070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3148157a40 0x7f31480c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 --2- 172.21.15.94:0/3922117922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c0039e0 0x7f314815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:48.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.976+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 >> 172.21.15.94:0/3922117922 conn(0x7f31480d1190 msgr2=0x7f31480cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:48.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.977+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 shutdown_connections 2024-05-27T12:35:48.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:48.977+0000 7f314e800640 1 -- 172.21.15.94:0/3922117922 wait complete. 2024-05-27T12:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.191+0000 7fa10c000640 1 Processor -- start 2024-05-27T12:35:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.191+0000 7fa10c000640 1 -- start start 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.192+0000 7fa10c000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1040cfb40 0x7fa1040d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.192+0000 7fa10c000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1040d1d70 0x7fa1040b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.192+0000 7fa10c000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040d2180 con 0x7fa10415eb00 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.192+0000 7fa10c000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fa1040d5700 con 0x7fa1040cfb40 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.192+0000 7fa10c000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa10415eee0 con 0x7fa1040d1d70 2024-05-27T12:35:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1040cfb40 0x7fa1040d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fa1040cfb40 0x7fa1040d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37840/0 (socket says 172.21.15.94:37840) 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 -- 172.21.15.94:0/1860543522 learned_addr learned my addr 172.21.15.94:0/1860543522 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa10a000640 1 --2- 172.21.15.94:0/1860543522 >> v2:172.21.15.97:3300/0 conn(0x7fa1040d1d70 0x7fa1040b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 -- 172.21.15.94:0/1860543522 >> v2:172.21.15.97:3300/0 conn(0x7fa1040d1d70 msgr2=0x7fa1040b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 --2- 172.21.15.94:0/1860543522 >> v2:172.21.15.97:3300/0 conn(0x7fa1040d1d70 0x7fa1040b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 -- 172.21.15.94:0/1860543522 >> v1:172.21.15.97:6789/0 conn(0x7fa10415eb00 legacy=0x7fa1040d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.193+0000 7fa108c00640 1 -- 172.21.15.94:0/1860543522 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040d4660 con 0x7fa1040cfb40 2024-05-27T12:35:49.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.194+0000 7fa108c00640 1 --2- 172.21.15.94:0/1860543522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040cfb40 0x7fa1040d1830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b3a40 tx=0x7fa0f002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fb11cacab0cc83 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.194+0000 7fa103e00640 1 -- 172.21.15.94:0/1860543522 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0037070 con 0x7fa1040cfb40 2024-05-27T12:35:49.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.194+0000 7fa103e00640 1 -- 172.21.15.94:0/1860543522 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0032440 con 0x7fa1040cfb40 2024-05-27T12:35:49.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.194+0000 7fa103e00640 1 -- 172.21.15.94:0/1860543522 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0032810 con 0x7fa1040cfb40 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 -- 172.21.15.94:0/1860543522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040cfb40 msgr2=0x7fa1040d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 --2- 172.21.15.94:0/1860543522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040cfb40 0x7fa1040d1830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b3a40 tx=0x7fa0f002a2c0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 -- 172.21.15.94:0/1860543522 shutdown_connections 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 --2- 172.21.15.94:0/1860543522 >> v2:172.21.15.97:3300/0 conn(0x7fa1040d1d70 0x7fa1040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 --2- 172.21.15.94:0/1860543522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040cfb40 0x7fa1040d1830 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 -- 172.21.15.94:0/1860543522 >> 172.21.15.94:0/1860543522 conn(0x7fa104067a50 msgr2=0x7fa104067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 -- 172.21.15.94:0/1860543522 shutdown_connections 2024-05-27T12:35:49.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.195+0000 7fa10c000640 1 -- 172.21.15.94:0/1860543522 wait complete. 2024-05-27T12:35:49.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.197+0000 7fa10c000640 1 Processor -- start 2024-05-27T12:35:49.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.197+0000 7fa10c000640 1 -- start start 2024-05-27T12:35:49.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1040cfb40 0x7fa104161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 0x7fa1040cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d4f00 con 0x7fa1040d1d70 2024-05-27T12:35:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d4c20 con 0x7fa1040cfb40 2024-05-27T12:35:49.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.198+0000 7fa10c000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040d51e0 con 0x7fa10415eb00 2024-05-27T12:35:49.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa10a000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 0x7fa1040cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37852/0 (socket says 172.21.15.94:37852) 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 -- 172.21.15.94:0/3432202839 learned_addr learned my addr 172.21.15.94:0/3432202839 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa109600640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1040cfb40 0x7fa104161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 -- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 msgr2=0x7fa1040cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 0x7fa1040cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 -- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1040cfb40 msgr2=0x7fa104161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1040cfb40 0x7fa104161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 -- 172.21.15.94:0/3432202839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10414c540 con 0x7fa1040d1d70 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa10a000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 0x7fa1040cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.199+0000 7fa108c00640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.200+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0041020 con 0x7fa1040d1d70 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.200+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0030970 con 0x7fa1040d1d70 2024-05-27T12:35:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.200+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0037040 con 0x7fa1040d1d70 2024-05-27T12:35:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.200+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1040ce290 con 0x7fa1040d1d70 2024-05-27T12:35:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.200+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1040cd4d0 con 0x7fa1040d1d70 2024-05-27T12:35:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.201+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa104068a70 con 0x7fa1040d1d70 2024-05-27T12:35:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.202+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0f0044020 con 0x7fa1040d1d70 2024-05-27T12:35:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.203+0000 7fa101600640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 0x7fa0e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.203+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa0f00ac5b0 con 0x7fa1040d1d70 2024-05-27T12:35:49.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.204+0000 7fa109600640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 0x7fa0e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.205+0000 7fa109600640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 0x7fa0e406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006fd0 tx=0x7fa0f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.207+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0f004b030 con 0x7fa1040d1d70 2024-05-27T12:35:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.407+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1040008d0 con 0x7fa1040d1d70 2024-05-27T12:35:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.408+0000 7fa101600640 1 -- 172.21.15.94:0/3432202839 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa1040d2180 con 0x7fa1040d1d70 2024-05-27T12:35:49.407 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 msgr2=0x7fa0e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 0x7fa0e406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006fd0 tx=0x7fa0f8008040 comp rx=0 tx=0).stop 2024-05-27T12:35:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 msgr2=0x7fa104161ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f00046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 shutdown_connections 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa0e406c8f0 0x7fa0e406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa10415eb00 0x7fa1040cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1040d1d70 0x7fa104161ec0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 --2- 172.21.15.94:0/3432202839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1040cfb40 0x7fa104161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 >> 172.21.15.94:0/3432202839 conn(0x7fa104067a50 msgr2=0x7fa1040b7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 shutdown_connections 2024-05-27T12:35:49.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.411+0000 7fa10c000640 1 -- 172.21.15.94:0/3432202839 wait complete. 2024-05-27T12:35:49.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T12:35:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.631+0000 7ff88a600640 1 Processor -- start 2024-05-27T12:35:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.632+0000 7ff88a600640 1 -- start start 2024-05-27T12:35:49.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.633+0000 7ff88a600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff884144e70 0x7ff8840cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.633+0000 7ff88a600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8840cdd80 con 0x7ff8840b4670 2024-05-27T12:35:49.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.633+0000 7ff88a600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8840d01a0 con 0x7ff8840cfd20 2024-05-27T12:35:49.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.633+0000 7ff88a600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff884145250 con 0x7ff884144e70 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.634+0000 7ff889c00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7ff8840b4670 0x7ff88406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51418/0 (socket says 172.21.15.94:51418) 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.634+0000 7ff889c00640 1 -- 172.21.15.94:0/736595338 learned_addr learned my addr 172.21.15.94:0/736595338 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.634+0000 7ff889200640 1 --2- 172.21.15.94:0/736595338 >> v2:172.21.15.97:3300/0 conn(0x7ff884144e70 0x7ff8840cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.634+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986552205 0 0) 0x7ff8840cdd80 con 0x7ff8840b4670 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.634+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff868003520 con 0x7ff8840b4670 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 959919454 0 0) 0x7ff868003520 con 0x7ff8840b4670 2024-05-27T12:35:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 >> v2:172.21.15.97:3300/0 conn(0x7ff884144e70 msgr2=0x7ff8840cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 --2- 172.21.15.94:0/736595338 >> v2:172.21.15.97:3300/0 conn(0x7ff884144e70 0x7ff8840cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 >> v1:172.21.15.97:6789/0 conn(0x7ff8840cfd20 legacy=0x7ff88415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840cd7c0 con 0x7ff8840b4670 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7ff87c001670 con 0x7ff8840b4670 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 >> v1:172.21.15.94:6789/0 conn(0x7ff8840b4670 legacy=0x7ff88406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff868003a00 0x7ff868003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff884145250 con 0x7ff8680051c0 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff86800c0c0 con 0x7ff868004800 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff86800c2c0 con 0x7ff868003a00 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff889c00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff889200640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff868003a00 0x7ff868003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff889c00640 1 -- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff868003a00 msgr2=0x7ff868003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff889c00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff868003a00 0x7ff868003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff883e00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.635+0000 7ff889c00640 1 -- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff868004800 msgr2=0x7ff868004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.636+0000 7ff889c00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.636+0000 7ff889c00640 1 -- 172.21.15.94:0/736595338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff868003520 con 0x7ff8680051c0 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.636+0000 7ff889200640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff868003a00 0x7ff868003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.636+0000 7ff883e00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.636+0000 7ff889c00640 1 -- 172.21.15.94:0/736595338 reap_dead start 2024-05-27T12:35:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff889c00640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00c0e0 tx=0x7ff868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320473b0bc7dbac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c007c40 con 0x7ff8680051c0 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c005af0 con 0x7ff8680051c0 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff883400640 1 -- 172.21.15.94:0/736595338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c007070 con 0x7ff8680051c0 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff88a600640 1 -- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 msgr2=0x7ff86800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff88a600640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00c0e0 tx=0x7ff868005830 comp rx=0 tx=0).stop 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff88a600640 1 -- 172.21.15.94:0/736595338 shutdown_connections 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff88a600640 1 --2- 172.21.15.94:0/736595338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.637+0000 7ff88a600640 1 -- 172.21.15.94:0/736595338 >> 172.21.15.94:0/736595338 conn(0x7ff88406b930 msgr2=0x7ff88406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.638+0000 7ff88a600640 1 -- 172.21.15.94:0/736595338 shutdown_connections 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.638+0000 7ff88a600640 1 -- 172.21.15.94:0/736595338 wait complete. 2024-05-27T12:35:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.639+0000 7ff88a600640 1 Processor -- start 2024-05-27T12:35:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.640+0000 7ff88a600640 1 -- start start 2024-05-27T12:35:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.640+0000 7ff88a600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8680051c0 0x7ff8840c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.640+0000 7ff88a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.640+0000 7ff88a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8840c7c50 0x7ff8840c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.640+0000 7ff88a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff88415f150 con 0x7ff88413ade0 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff88a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff88415f4c0 con 0x7ff8840c7c50 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff88a600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff8840cd0c0 con 0x7ff8680051c0 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37872/0 (socket says 172.21.15.94:37872) 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 -- 172.21.15.94:0/4024341877 learned_addr learned my addr 172.21.15.94:0/4024341877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff889200640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8680051c0 0x7ff8840c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff889c00640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8840c7c50 0x7ff8840c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 -- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8680051c0 msgr2=0x7ff8840c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8680051c0 0x7ff8840c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 -- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8840c7c50 msgr2=0x7ff8840c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8840c7c50 0x7ff8840c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.641+0000 7ff883e00640 1 -- 172.21.15.94:0/4024341877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88415c7e0 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff883e00640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d5a0 tx=0x7ff87000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870005480 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff870017070 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870003850 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88415d820 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.642+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88415ca60 con 0x7ff88413ade0 2024-05-27T12:35:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.643+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884075460 con 0x7ff88413ade0 2024-05-27T12:35:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.644+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8700039f0 con 0x7ff88413ade0 2024-05-27T12:35:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.645+0000 7ff880c00640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.645+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff87008b980 con 0x7ff88413ade0 2024-05-27T12:35:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.646+0000 7ff889200640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.647+0000 7ff889200640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 0x7ff86406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff874003be0 tx=0x7ff87400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.650+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff88415c7e0 con 0x7ff88413ade0 2024-05-27T12:35:49.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.861+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8841f3b30 con 0x7ff88413ade0 2024-05-27T12:35:49.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.862+0000 7ff880c00640 1 -- 172.21.15.94:0/4024341877 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff87008b070 con 0x7ff88413ade0 2024-05-27T12:35:49.860 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 msgr2=0x7ff86406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 0x7ff86406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff874003be0 tx=0x7ff87400b040 comp rx=0 tx=0).stop 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 msgr2=0x7ff8841f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d5a0 tx=0x7ff87000da60 comp rx=0 tx=0).stop 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 shutdown_connections 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8840c7c50 0x7ff8840c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff88413ade0 0x7ff8841f35f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 --2- 172.21.15.94:0/4024341877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8680051c0 0x7ff8840c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.864+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 >> 172.21.15.94:0/4024341877 conn(0x7ff88406b930 msgr2=0x7ff884071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.865+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 shutdown_connections 2024-05-27T12:35:49.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:49.865+0000 7ff88a600640 1 -- 172.21.15.94:0/4024341877 wait complete. 2024-05-27T12:35:49.883 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-27T12:35:49.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:49.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-27T12:35:49.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-05-27T12:35:49.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:35:49.978 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-27T12:35:50.028 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-27T12:35:50.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:50.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.238+0000 7f50d2c00640 1 Processor -- start 2024-05-27T12:35:50.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.238+0000 7f50d2c00640 1 -- start start 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d2c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f50cc13e600 0x7f50cc071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d2c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50cc0cdb90 0x7f50cc06f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d2c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50cc13e9e0 con 0x7f50cc06eed0 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d2c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc15ba50 con 0x7f50cc13e600 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d2c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc15bd10 con 0x7f50cc0cdb90 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f50cc13e600 0x7f50cc071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f50cc13e600 0x7f50cc071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38604/0 (socket says 172.21.15.94:38604) 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 -- 172.21.15.94:0/3642324505 learned_addr learned my addr 172.21.15.94:0/3642324505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 -- 172.21.15.94:0/3642324505 >> v2:172.21.15.97:3300/0 conn(0x7f50cc0cdb90 msgr2=0x7f50cc06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 --2- 172.21.15.94:0/3642324505 >> v2:172.21.15.97:3300/0 conn(0x7f50cc0cdb90 0x7f50cc06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.240+0000 7f50d1800640 1 -- 172.21.15.94:0/3642324505 >> v1:172.21.15.97:6789/0 conn(0x7f50cc06eed0 legacy=0x7f50cc0cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.241+0000 7f50d1800640 1 -- 172.21.15.94:0/3642324505 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc15bff0 con 0x7f50cc13e600 2024-05-27T12:35:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.242+0000 7f50d1800640 1 --2- 172.21.15.94:0/3642324505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13e600 0x7f50cc071d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0aff20 tx=0x7f50c002c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59b6233822d84dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.242+0000 7f50cbe00640 1 -- 172.21.15.94:0/3642324505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0038070 con 0x7f50cc13e600 2024-05-27T12:35:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.242+0000 7f50cbe00640 1 -- 172.21.15.94:0/3642324505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0031710 con 0x7f50cc13e600 2024-05-27T12:35:50.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.242+0000 7f50cbe00640 1 -- 172.21.15.94:0/3642324505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0031ae0 con 0x7f50cc13e600 2024-05-27T12:35:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 -- 172.21.15.94:0/3642324505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13e600 msgr2=0x7f50cc071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 --2- 172.21.15.94:0/3642324505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13e600 0x7f50cc071d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0aff20 tx=0x7f50c002c440 comp rx=0 tx=0).stop 2024-05-27T12:35:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 -- 172.21.15.94:0/3642324505 shutdown_connections 2024-05-27T12:35:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 --2- 172.21.15.94:0/3642324505 >> v2:172.21.15.97:3300/0 conn(0x7f50cc0cdb90 0x7f50cc06f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 --2- 172.21.15.94:0/3642324505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13e600 0x7f50cc071d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 -- 172.21.15.94:0/3642324505 >> 172.21.15.94:0/3642324505 conn(0x7f50cc0b2160 msgr2=0x7f50cc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 -- 172.21.15.94:0/3642324505 shutdown_connections 2024-05-27T12:35:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.243+0000 7f50d2c00640 1 -- 172.21.15.94:0/3642324505 wait complete. 2024-05-27T12:35:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.244+0000 7f50d2c00640 1 Processor -- start 2024-05-27T12:35:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 -- start start 2024-05-27T12:35:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50cc06eed0 0x7f50cc13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50cc13e600 0x7f50cc15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc15c920 con 0x7f50cc0cdb90 2024-05-27T12:35:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc15c5b0 con 0x7f50cc13e600 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.245+0000 7f50d2c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc141030 con 0x7f50cc06eed0 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38616/0 (socket says 172.21.15.94:38616) 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 -- 172.21.15.94:0/1825688931 learned_addr learned my addr 172.21.15.94:0/1825688931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d2200640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50cc13e600 0x7f50cc15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 -- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50cc06eed0 msgr2=0x7f50cc13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50cc06eed0 0x7f50cc13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 -- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50cc13e600 msgr2=0x7f50cc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50cc13e600 0x7f50cc15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 -- 172.21.15.94:0/1825688931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc1f5400 con 0x7f50cc0cdb90 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.246+0000 7f50d0e00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c410 tx=0x7f50b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.247+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8017070 con 0x7f50cc0cdb90 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.247+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b80035f0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.247+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8003a20 con 0x7f50cc0cdb90 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.247+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1f56e0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.247+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1e3ed0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.248+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc075460 con 0x7f50cc0cdb90 2024-05-27T12:35:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.249+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50b801a070 con 0x7f50cc0cdb90 2024-05-27T12:35:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.250+0000 7f50c9600640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 0x7f50ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.250+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f50b808b0f0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.251+0000 7f50d1800640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 0x7f50ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.252+0000 7f50d1800640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 0x7f50ac06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c0035000 tx=0x7f50c0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.255+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50b8059490 con 0x7f50cc0cdb90 2024-05-27T12:35:50.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.477+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50cc1545f0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.478+0000 7f50c9600640 1 -- 172.21.15.94:0/1825688931 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f50b80010c0 con 0x7f50cc0cdb90 2024-05-27T12:35:50.476 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:50.476 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:50.477 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 msgr2=0x7f50ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 0x7f50ac06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c0035000 tx=0x7f50c0030000 comp rx=0 tx=0).stop 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 msgr2=0x7f50cc1e46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c410 tx=0x7f50b800c8d0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 shutdown_connections 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f50ac06c8f0 0x7f50ac06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50cc13e600 0x7f50cc15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc0cdb90 0x7f50cc1e46b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 --2- 172.21.15.94:0/1825688931 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50cc06eed0 0x7f50cc13d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 >> 172.21.15.94:0/1825688931 conn(0x7f50cc0b2160 msgr2=0x7f50cc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.480+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 shutdown_connections 2024-05-27T12:35:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.481+0000 7f50d2c00640 1 -- 172.21.15.94:0/1825688931 wait complete. 2024-05-27T12:35:50.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:50.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.696+0000 7fbfcf000640 1 Processor -- start 2024-05-27T12:35:50.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.696+0000 7fbfcf000640 1 -- start start 2024-05-27T12:35:50.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.697+0000 7fbfcf000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbfc8149160 0x7fbfc80ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcf000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfc8072fb0 0x7fbfc8071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcf000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc806dfc0 con 0x7fbfc8072460 2024-05-27T12:35:50.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcf000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc806ed10 con 0x7fbfc8149160 2024-05-27T12:35:50.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcf000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc8149540 con 0x7fbfc8072fb0 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfc7e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbfc8149160 0x7fbfc80ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcd000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbfc8072460 0x7fbfc80b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54054/0 (socket says 172.21.15.94:54054) 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfcd000640 1 -- 172.21.15.94:0/2418028038 learned_addr learned my addr 172.21.15.94:0/2418028038 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfbf400640 1 --2- 172.21.15.94:0/2418028038 >> v2:172.21.15.97:3300/0 conn(0x7fbfc8072fb0 0x7fbfc8071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfc7e00640 1 -- 172.21.15.94:0/2418028038 >> v2:172.21.15.97:3300/0 conn(0x7fbfc8072fb0 msgr2=0x7fbfc8071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfc7e00640 1 --2- 172.21.15.94:0/2418028038 >> v2:172.21.15.97:3300/0 conn(0x7fbfc8072fb0 0x7fbfc8071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfc7e00640 1 -- 172.21.15.94:0/2418028038 >> v1:172.21.15.94:6789/0 conn(0x7fbfc8072460 legacy=0x7fbfc80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.698+0000 7fbfc7400640 1 -- 172.21.15.94:0/2418028038 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372181487 0 0) 0x7fbfc806dfc0 con 0x7fbfc8072460 2024-05-27T12:35:50.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.699+0000 7fbfc7e00640 1 -- 172.21.15.94:0/2418028038 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8072880 con 0x7fbfc8149160 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfc7e00640 1 --2- 172.21.15.94:0/2418028038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8149160 0x7fbfc80ce610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806f1f0 tx=0x7fbfb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600c32dd018f75a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfc7400640 1 -- 172.21.15.94:0/2418028038 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8007740 con 0x7fbfc8149160 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfc7400640 1 -- 172.21.15.94:0/2418028038 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb80078e0 con 0x7fbfc8149160 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfc7400640 1 -- 172.21.15.94:0/2418028038 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8007cb0 con 0x7fbfc8149160 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfcf000640 1 -- 172.21.15.94:0/2418028038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8149160 msgr2=0x7fbfc80ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfcf000640 1 --2- 172.21.15.94:0/2418028038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8149160 0x7fbfc80ce610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806f1f0 tx=0x7fbfb802a680 comp rx=0 tx=0).stop 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfcf000640 1 -- 172.21.15.94:0/2418028038 shutdown_connections 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.700+0000 7fbfcf000640 1 --2- 172.21.15.94:0/2418028038 >> v2:172.21.15.97:3300/0 conn(0x7fbfc8072fb0 0x7fbfc8071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.701+0000 7fbfcf000640 1 --2- 172.21.15.94:0/2418028038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8149160 0x7fbfc80ce610 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.701+0000 7fbfcf000640 1 -- 172.21.15.94:0/2418028038 >> 172.21.15.94:0/2418028038 conn(0x7fbfc80b3830 msgr2=0x7fbfc80b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:50.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.701+0000 7fbfcf000640 1 -- 172.21.15.94:0/2418028038 shutdown_connections 2024-05-27T12:35:50.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.701+0000 7fbfcf000640 1 -- 172.21.15.94:0/2418028038 wait complete. 2024-05-27T12:35:50.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.702+0000 7fbfcf000640 1 Processor -- start 2024-05-27T12:35:50.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.702+0000 7fbfcf000640 1 -- start start 2024-05-27T12:35:50.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbfc8072460 0x7fbfc8163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfc8149160 0x7fbfc8143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80d03a0 con 0x7fbfc8140c50 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc806e510 con 0x7fbfc8149160 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.703+0000 7fbfcf000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80cff30 con 0x7fbfc8072460 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38632/0 (socket says 172.21.15.94:38632) 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 -- 172.21.15.94:0/3422711664 learned_addr learned my addr 172.21.15.94:0/3422711664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfc7e00640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbfc8072460 0x7fbfc8163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfbf400640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfc8149160 0x7fbfc8143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 -- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbfc8072460 msgr2=0x7fbfc8163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbfc8072460 0x7fbfc8163ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 -- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfc8149160 msgr2=0x7fbfc8143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfc8149160 0x7fbfc8143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.704+0000 7fbfcd000640 1 -- 172.21.15.94:0/3422711664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8156450 con 0x7fbfc8140c50 2024-05-27T12:35:50.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfcd000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c450 tx=0x7fbfc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00074e0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc00047e0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0004bd0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81633f0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.705+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81639b0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.706+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc80cefc0 con 0x7fbfc8140c50 2024-05-27T12:35:50.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.707+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfc0011070 con 0x7fbfc8140c50 2024-05-27T12:35:50.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.708+0000 7fbfc4c00640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 0x7fbfa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:50.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.708+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbfc008b6e0 con 0x7fbfc8140c50 2024-05-27T12:35:50.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.709+0000 7fbfc7e00640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 0x7fbfa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:50.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.710+0000 7fbfc7e00640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 0x7fbfa406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8034000 tx=0x7fbfb8033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:50.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.712+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfc80cefc0 con 0x7fbfc8140c50 2024-05-27T12:35:50.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.917+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfc8075460 con 0x7fbfc8140c50 2024-05-27T12:35:50.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.920+0000 7fbfc4c00640 1 -- 172.21.15.94:0/3422711664 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7fbfc0059a90 con 0x7fbfc8140c50 2024-05-27T12:35:50.919 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:50.919 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:46.194696+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 msgr2=0x7fbfa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 0x7fbfa406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8034000 tx=0x7fbfb8033040 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 msgr2=0x7fbfc81410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c450 tx=0x7fbfc000c910 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 shutdown_connections 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbfa406c920 0x7fbfa406ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfc8149160 0x7fbfc8143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfc8140c50 0x7fbfc81410e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 --2- 172.21.15.94:0/3422711664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbfc8072460 0x7fbfc8163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 >> 172.21.15.94:0/3422711664 conn(0x7fbfc80b3830 msgr2=0x7fbfc80b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 shutdown_connections 2024-05-27T12:35:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:50.924+0000 7fbfcf000640 1 -- 172.21.15.94:0/3422711664 wait complete. 2024-05-27T12:35:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T12:34:56.888194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T12:34:58.330763+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:35:50.942 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:35:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.145+0000 7f07d2600640 1 Processor -- start 2024-05-27T12:35:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.145+0000 7f07d2600640 1 -- start start 2024-05-27T12:35:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.147+0000 7f07d2600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f07cc072430 0x7f07cc1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.147+0000 7f07d2600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07cc0b3d70 con 0x7f07cc06ed40 2024-05-27T12:35:51.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.147+0000 7f07d2600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07cc0b4050 con 0x7f07cc145050 2024-05-27T12:35:51.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.147+0000 7f07d2600640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f07cc0b4330 con 0x7f07cc072430 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07d1c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f07cc072430 0x7f07cc1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07cbe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f07cc06ed40 0x7f07cc071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54064/0 (socket says 172.21.15.94:54064) 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07cbe00640 1 -- 172.21.15.94:0/4074531517 learned_addr learned my addr 172.21.15.94:0/4074531517 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07d1c00640 1 -- 172.21.15.94:0/4074531517 >> v1:172.21.15.97:6789/0 conn(0x7f07cc145050 legacy=0x7f07cc0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07d1c00640 1 -- 172.21.15.94:0/4074531517 >> v1:172.21.15.94:6789/0 conn(0x7f07cc06ed40 legacy=0x7f07cc071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07cb400640 1 -- 172.21.15.94:0/4074531517 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659084287 0 0) 0x7f07cc0b3d70 con 0x7f07cc06ed40 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.148+0000 7f07d1c00640 1 -- 172.21.15.94:0/4074531517 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc0b4610 con 0x7f07cc072430 2024-05-27T12:35:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.149+0000 7f07d1c00640 1 --2- 172.21.15.94:0/4074531517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc072430 0x7f07cc1455e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009d90 tx=0x7f07c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1a479b30411d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07cb400640 1 -- 172.21.15.94:0/4074531517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4007740 con 0x7f07cc072430 2024-05-27T12:35:51.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07cb400640 1 -- 172.21.15.94:0/4074531517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c40078e0 con 0x7f07cc072430 2024-05-27T12:35:51.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07cb400640 1 -- 172.21.15.94:0/4074531517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4007cb0 con 0x7f07cc072430 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 -- 172.21.15.94:0/4074531517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc072430 msgr2=0x7f07cc1455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 --2- 172.21.15.94:0/4074531517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc072430 0x7f07cc1455e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009d90 tx=0x7f07c402a660 comp rx=0 tx=0).stop 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 -- 172.21.15.94:0/4074531517 shutdown_connections 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 --2- 172.21.15.94:0/4074531517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc072430 0x7f07cc1455e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 -- 172.21.15.94:0/4074531517 >> 172.21.15.94:0/4074531517 conn(0x7f07cc0b6230 msgr2=0x7f07cc0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.150+0000 7f07d2600640 1 -- 172.21.15.94:0/4074531517 shutdown_connections 2024-05-27T12:35:51.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.151+0000 7f07d2600640 1 -- 172.21.15.94:0/4074531517 wait complete. 2024-05-27T12:35:51.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.152+0000 7f07d2600640 1 Processor -- start 2024-05-27T12:35:51.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.152+0000 7f07d2600640 1 -- start start 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.153+0000 7f07d2600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc06ed40 0x7f07cc140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.153+0000 7f07d2600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07cc072430 0x7f07cc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.153+0000 7f07d2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0af280 con 0x7f07cc145050 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d2600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0af100 con 0x7f07cc06ed40 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d2600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc0af400 con 0x7f07cc072430 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38640/0 (socket says 172.21.15.94:38640) 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 -- 172.21.15.94:0/607554240 learned_addr learned my addr 172.21.15.94:0/607554240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1200640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc06ed40 0x7f07cc140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07cbe00640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07cc072430 0x7f07cc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 -- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07cc072430 msgr2=0x7f07cc155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07cc072430 0x7f07cc155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 -- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc06ed40 msgr2=0x7f07cc140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc06ed40 0x7f07cc140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.154+0000 7f07d1c00640 1 -- 172.21.15.94:0/607554240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc0c88a0 con 0x7f07cc145050 2024-05-27T12:35:51.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07d1c00640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009eb0 tx=0x7f07c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4040070 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4031c90 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc0c8b80 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4045680 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.155+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc0c9140 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.156+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc075460 con 0x7f07cc145050 2024-05-27T12:35:51.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.157+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07c4039040 con 0x7f07cc145050 2024-05-27T12:35:51.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.158+0000 7f07c8c00640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.159+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f07c40ad5c0 con 0x7f07cc145050 2024-05-27T12:35:51.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.159+0000 7f07d1200640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.160+0000 7f07d1200640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 0x7f07ac06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.163+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07c4043360 con 0x7f07cc145050 2024-05-27T12:35:51.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.361+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07cc1438f0 con 0x7f07cc145050 2024-05-27T12:35:51.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.362+0000 7f07c8c00640 1 -- 172.21.15.94:0/607554240 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f07c407ba20 con 0x7f07cc145050 2024-05-27T12:35:51.360 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:35:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.364+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 msgr2=0x7f07ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.364+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 0x7f07ac06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0009040 comp rx=0 tx=0).stop 2024-05-27T12:35:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.364+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 msgr2=0x7f07cc141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.364+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009eb0 tx=0x7f07c4004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 shutdown_connections 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f07ac06c8f0 0x7f07ac06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07cc145050 0x7f07cc141110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07cc072430 0x7f07cc155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 --2- 172.21.15.94:0/607554240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc06ed40 0x7f07cc140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 >> 172.21.15.94:0/607554240 conn(0x7f07cc0b6230 msgr2=0x7f07cc0ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 shutdown_connections 2024-05-27T12:35:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.365+0000 7f07d2600640 1 -- 172.21.15.94:0/607554240 wait complete. 2024-05-27T12:35:51.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T12:35:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.581+0000 7ff41e200640 1 Processor -- start 2024-05-27T12:35:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.581+0000 7ff41e200640 1 -- start start 2024-05-27T12:35:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.583+0000 7ff41e200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff418146060 0x7ff4180b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.583+0000 7ff41e200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4180cd8a0 0x7ff41806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.583+0000 7ff41e200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41815cfd0 con 0x7ff4180cdd00 2024-05-27T12:35:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.583+0000 7ff41e200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff41815dbe0 con 0x7ff418146060 2024-05-27T12:35:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.583+0000 7ff41e200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff41815d2b0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff418146060 0x7ff4180b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff418146060 0x7ff4180b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:47898/0 (socket says 172.21.15.94:47898) 2024-05-27T12:35:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff416a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4180cd8a0 0x7ff41806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 -- 172.21.15.94:0/2393931127 learned_addr learned my addr 172.21.15.94:0/2393931127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 -- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4180cd8a0 msgr2=0x7ff41806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 --2- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4180cd8a0 0x7ff41806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 -- 172.21.15.94:0/2393931127 >> v1:172.21.15.97:6789/0 conn(0x7ff4180cdd00 legacy=0x7ff418072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.584+0000 7ff417400640 1 -- 172.21.15.94:0/2393931127 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41815d590 con 0x7ff418146060 2024-05-27T12:35:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.586+0000 7ff417400640 1 --2- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff418146060 0x7ff4180b3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff408009870 tx=0x7ff40802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73c1f4f3b6984a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.586+0000 7ff416000640 1 -- 172.21.15.94:0/2393931127 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408037070 con 0x7ff418146060 2024-05-27T12:35:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.586+0000 7ff416000640 1 -- 172.21.15.94:0/2393931127 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff408032440 con 0x7ff418146060 2024-05-27T12:35:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.586+0000 7ff416000640 1 -- 172.21.15.94:0/2393931127 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408032810 con 0x7ff418146060 2024-05-27T12:35:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 -- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff418146060 msgr2=0x7ff4180b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 --2- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff418146060 0x7ff4180b3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff408009870 tx=0x7ff40802a250 comp rx=0 tx=0).stop 2024-05-27T12:35:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 -- 172.21.15.94:0/2393931127 shutdown_connections 2024-05-27T12:35:51.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 --2- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4180cd8a0 0x7ff41806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 --2- 172.21.15.94:0/2393931127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff418146060 0x7ff4180b3f70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 -- 172.21.15.94:0/2393931127 >> 172.21.15.94:0/2393931127 conn(0x7ff4180b2240 msgr2=0x7ff4180b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:51.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 -- 172.21.15.94:0/2393931127 shutdown_connections 2024-05-27T12:35:51.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.587+0000 7ff41e200640 1 -- 172.21.15.94:0/2393931127 wait complete. 2024-05-27T12:35:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.589+0000 7ff41e200640 1 Processor -- start 2024-05-27T12:35:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.589+0000 7ff41e200640 1 -- start start 2024-05-27T12:35:51.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 0x7ff41815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff418146060 0x7ff418137830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4180d0110 con 0x7ff4180cd8a0 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff418146440 con 0x7ff4180cdd00 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff41e200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff41806be70 con 0x7ff418146060 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff417400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff417400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38644/0 (socket says 172.21.15.94:38644) 2024-05-27T12:35:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.590+0000 7ff417400640 1 -- 172.21.15.94:0/204617114 learned_addr learned my addr 172.21.15.94:0/204617114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff416a00640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 0x7ff41815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417e00640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff418146060 0x7ff418137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 -- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff418146060 msgr2=0x7ff418137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff418146060 0x7ff418137830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 -- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 msgr2=0x7ff41815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 0x7ff41815bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 -- 172.21.15.94:0/204617114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4181458a0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff416a00640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 0x7ff41815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff417400640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff408009990 tx=0x7ff4080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408037040 con 0x7ff4180cd8a0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.591+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff408042070 con 0x7ff4180cd8a0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.592+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408030940 con 0x7ff4180cd8a0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.592+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4180c59f0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.592+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4181452e0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.593+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff418068a70 con 0x7ff4180cd8a0 2024-05-27T12:35:51.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.594+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff408030ae0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.594+0000 7ff40f400640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.595+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff4080ac570 con 0x7ff4180cd8a0 2024-05-27T12:35:51.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.595+0000 7ff416a00640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:51.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.596+0000 7ff416a00640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 0x7ff3f806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4180b2210 tx=0x7ff404008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:51.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.599+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff40807a7f0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.802+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4180b3840 con 0x7ff4180cd8a0 2024-05-27T12:35:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.804+0000 7ff40f400640 1 -- 172.21.15.94:0/204617114 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff40807a0f0 con 0x7ff4180cd8a0 2024-05-27T12:35:51.802 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 msgr2=0x7ff3f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 0x7ff3f806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4180b2210 tx=0x7ff404008040 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 msgr2=0x7ff4180c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff408009990 tx=0x7ff4080046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 shutdown_connections 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4180cdd00 0x7ff41815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff418146060 0x7ff418137830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 --2- 172.21.15.94:0/204617114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4180cd8a0 0x7ff4180c54b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 >> 172.21.15.94:0/204617114 conn(0x7ff4180b2240 msgr2=0x7ff4180b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 shutdown_connections 2024-05-27T12:35:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:51.806+0000 7ff41e200640 1 -- 172.21.15.94:0/204617114 wait complete. 2024-05-27T12:35:51.823 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T12:35:52.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.018+0000 7fb226c00640 1 Processor -- start 2024-05-27T12:35:52.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.019+0000 7fb226c00640 1 -- start start 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb226c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb220067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb226c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2200ba3f0 0x7fb2200b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb226c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200b7840 con 0x7fb2200b66d0 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb226c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb220067f30 con 0x7fb2200ba3f0 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb226c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb220068210 con 0x7fb22015e900 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb224e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2200ba3f0 0x7fb2200b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb224e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb2200ba3f0 0x7fb2200b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38656/0 (socket says 172.21.15.94:38656) 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.020+0000 7fb224e00640 1 -- 172.21.15.94:0/3282242980 learned_addr learned my addr 172.21.15.94:0/3282242980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.021+0000 7fb225800640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb220067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.021+0000 7fb224e00640 1 -- 172.21.15.94:0/3282242980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 msgr2=0x7fb220067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.021+0000 7fb224e00640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb220067630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.021+0000 7fb224e00640 1 -- 172.21.15.94:0/3282242980 >> v1:172.21.15.97:6789/0 conn(0x7fb2200b66d0 legacy=0x7fb220162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.021+0000 7fb224e00640 1 -- 172.21.15.94:0/3282242980 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200684f0 con 0x7fb2200ba3f0 2024-05-27T12:35:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.022+0000 7fb224e00640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb2200b6190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e50 tx=0x7fb21002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffba065e1d2eeb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.022+0000 7fb21fe00640 1 -- 172.21.15.94:0/3282242980 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210036070 con 0x7fb2200ba3f0 2024-05-27T12:35:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.022+0000 7fb21fe00640 1 -- 172.21.15.94:0/3282242980 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210031600 con 0x7fb2200ba3f0 2024-05-27T12:35:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.022+0000 7fb21fe00640 1 -- 172.21.15.94:0/3282242980 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100319d0 con 0x7fb2200ba3f0 2024-05-27T12:35:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 -- 172.21.15.94:0/3282242980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 msgr2=0x7fb2200b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb2200b6190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e50 tx=0x7fb21002a750 comp rx=0 tx=0).stop 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 -- 172.21.15.94:0/3282242980 shutdown_connections 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb2200b6190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 --2- 172.21.15.94:0/3282242980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb220067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 -- 172.21.15.94:0/3282242980 >> 172.21.15.94:0/3282242980 conn(0x7fb2200d1190 msgr2=0x7fb2200d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 -- 172.21.15.94:0/3282242980 shutdown_connections 2024-05-27T12:35:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.023+0000 7fb226c00640 1 -- 172.21.15.94:0/3282242980 wait complete. 2024-05-27T12:35:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.025+0000 7fb226c00640 1 Processor -- start 2024-05-27T12:35:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.025+0000 7fb226c00640 1 -- start start 2024-05-27T12:35:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b66d0 0x7fb220151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb22015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb22015ece0 con 0x7fb2200ba3f0 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200d2df0 con 0x7fb2200b66d0 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb226c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb220067b70 con 0x7fb22015e900 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb224e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb224e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38666/0 (socket says 172.21.15.94:38666) 2024-05-27T12:35:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.026+0000 7fb224e00640 1 -- 172.21.15.94:0/798400234 learned_addr learned my addr 172.21.15.94:0/798400234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 -- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 msgr2=0x7fb22015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb22015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 -- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b66d0 msgr2=0x7fb220151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b66d0 0x7fb220151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 -- 172.21.15.94:0/798400234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200cb980 con 0x7fb2200ba3f0 2024-05-27T12:35:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb224e00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e50 tx=0x7fb210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210036050 con 0x7fb2200ba3f0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb210042070 con 0x7fb2200ba3f0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200cbc60 con 0x7fb2200ba3f0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210009510 con 0x7fb2200ba3f0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.027+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200cc220 con 0x7fb2200ba3f0 2024-05-27T12:35:52.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.029+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220075460 con 0x7fb2200ba3f0 2024-05-27T12:35:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.029+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb210007c50 con 0x7fb2200ba3f0 2024-05-27T12:35:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.030+0000 7fb21d600640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.030+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb2100aca90 con 0x7fb2200ba3f0 2024-05-27T12:35:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.031+0000 7fb225800640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.032+0000 7fb225800640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 0x7fb20406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb2140098f0 tx=0x7fb214008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.034+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2200b7840 con 0x7fb2200ba3f0 2024-05-27T12:35:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.218+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb220153860 con 0x7fb2200ba3f0 2024-05-27T12:35:52.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.219+0000 7fb21d600640 1 -- 172.21.15.94:0/798400234 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fb21007aed0 con 0x7fb2200ba3f0 2024-05-27T12:35:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 msgr2=0x7fb20406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 0x7fb20406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb2140098f0 tx=0x7fb214008040 comp rx=0 tx=0).stop 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 msgr2=0x7fb22015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb210009e50 tx=0x7fb210004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 shutdown_connections 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb20406c8f0 0x7fb20406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb22015e900 0x7fb22015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2200ba3f0 0x7fb22015b150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 --2- 172.21.15.94:0/798400234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2200b66d0 0x7fb220151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.221+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 >> 172.21.15.94:0/798400234 conn(0x7fb2200d1190 msgr2=0x7fb220154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:52.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.222+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 shutdown_connections 2024-05-27T12:35:52.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.222+0000 7fb226c00640 1 -- 172.21.15.94:0/798400234 wait complete. 2024-05-27T12:35:52.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.427+0000 7f91f6000640 1 Processor -- start 2024-05-27T12:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.427+0000 7f91f6000640 1 -- start start 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.428+0000 7f91f6000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91f006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91f6000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f0160360 con 0x7f91f006f280 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91f6000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f0160640 con 0x7f91f00cfb10 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91f6000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f91f0160f70 con 0x7f91f015a250 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91eea00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f91f006f280 0x7f91f00cf400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54070/0 (socket says 172.21.15.94:54070) 2024-05-27T12:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91eea00640 1 -- 172.21.15.94:0/199333305 learned_addr learned my addr 172.21.15.94:0/199333305 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500689200 0 0) 0x7f91f0160360 con 0x7f91f006f280 2024-05-27T12:35:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91d4003520 con 0x7f91f006f280 2024-05-27T12:35:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.429+0000 7f91ef400640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91f006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1258690697 0 0) 0x7f91d4003520 con 0x7f91f006f280 2024-05-27T12:35:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 msgr2=0x7f91f006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91f006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 >> v1:172.21.15.97:6789/0 conn(0x7f91f00cfb10 legacy=0x7f91f00c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f0160920 con 0x7f91f006f280 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f91dc001670 con 0x7f91f006f280 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 >> v1:172.21.15.94:6789/0 conn(0x7f91f006f280 legacy=0x7f91f00cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91d4003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d40040d0 0x7f91d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d4004db0 0x7f91d4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91d400bbf0 con 0x7f91d40040d0 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f91d400bda0 con 0x7f91d4004db0 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f91d400bfa0 con 0x7f91f015a250 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91eea00640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d40040d0 0x7f91d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91ef400640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91d4003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91eea00640 1 -- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 msgr2=0x7f91d4003ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.430+0000 7f91eea00640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f015a250 0x7f91d4003ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.431+0000 7f91eea00640 1 -- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d4004db0 msgr2=0x7f91d4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.431+0000 7f91efe00640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d4004db0 0x7f91d4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.431+0000 7f91eea00640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d4004db0 0x7f91d4005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.431+0000 7f91eea00640 1 -- 172.21.15.94:0/199333305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4003520 con 0x7f91d40040d0 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91eea00640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d40040d0 0x7f91d4004800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00c240 tx=0x7f91d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a541efe782d5861f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc007d30 con 0x7f91d40040d0 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc005af0 con 0x7f91d40040d0 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91ee000640 1 -- 172.21.15.94:0/199333305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc007070 con 0x7f91d40040d0 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91f6000640 1 -- 172.21.15.94:0/199333305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d40040d0 msgr2=0x7f91d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91f6000640 1 --2- 172.21.15.94:0/199333305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d40040d0 0x7f91d4004800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00c240 tx=0x7f91d4005830 comp rx=0 tx=0).stop 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.432+0000 7f91efe00640 1 -- 172.21.15.94:0/199333305 reap_dead start 2024-05-27T12:35:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.433+0000 7f91f6000640 1 -- 172.21.15.94:0/199333305 shutdown_connections 2024-05-27T12:35:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.433+0000 7f91f6000640 1 -- 172.21.15.94:0/199333305 >> 172.21.15.94:0/199333305 conn(0x7f91f00b3830 msgr2=0x7f91f00b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.433+0000 7f91f6000640 1 -- 172.21.15.94:0/199333305 shutdown_connections 2024-05-27T12:35:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.433+0000 7f91f6000640 1 -- 172.21.15.94:0/199333305 wait complete. 2024-05-27T12:35:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.434+0000 7f91f6000640 1 Processor -- start 2024-05-27T12:35:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.435+0000 7f91f6000640 1 -- start start 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.435+0000 7f91f6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40040d0 0x7f91f01e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.435+0000 7f91f6000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f01f3830 0x7f91f01f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.435+0000 7f91f6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91f6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91f006e7f0 con 0x7f91f01f53d0 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91f6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f91f015a630 con 0x7f91d40040d0 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91f6000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f91f006e970 con 0x7f91f01f3830 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38682/0 (socket says 172.21.15.94:38682) 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 -- 172.21.15.94:0/3516537085 learned_addr learned my addr 172.21.15.94:0/3516537085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91eea00640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f01f3830 0x7f91f01f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91ef400640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40040d0 0x7f91f01e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 -- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f01f3830 msgr2=0x7f91f01f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f01f3830 0x7f91f01f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 -- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40040d0 msgr2=0x7f91f01e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40040d0 0x7f91f01e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.436+0000 7f91efe00640 1 -- 172.21.15.94:0/3516537085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f01f3df0 con 0x7f91f01f53d0 2024-05-27T12:35:52.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91efe00640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91e800d770 tx=0x7f91e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e801a070 con 0x7f91f01f53d0 2024-05-27T12:35:52.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e8004af0 con 0x7f91f01f53d0 2024-05-27T12:35:52.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800b6b0 con 0x7f91f01f53d0 2024-05-27T12:35:52.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01f40d0 con 0x7f91f01f53d0 2024-05-27T12:35:52.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.437+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01f4690 con 0x7f91f01f53d0 2024-05-27T12:35:52.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.438+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f0075460 con 0x7f91f01f53d0 2024-05-27T12:35:52.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.439+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91e800b850 con 0x7f91f01f53d0 2024-05-27T12:35:52.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.440+0000 7f91e7400640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.440+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f91e808ae70 con 0x7f91f01f53d0 2024-05-27T12:35:52.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.441+0000 7f91ef400640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.442+0000 7f91ef400640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 0x7f91d006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91e0006fd0 tx=0x7f91e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.444+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91e8059180 con 0x7f91f01f53d0 2024-05-27T12:35:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.683+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91f00c57b0 con 0x7f91f01f53d0 2024-05-27T12:35:52.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.685+0000 7f91e7400640 1 -- 172.21.15.94:0/3516537085 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f91e8058a80 con 0x7f91f01f53d0 2024-05-27T12:35:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:52.684 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:52.685 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 msgr2=0x7f91d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 0x7f91d006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91e0006fd0 tx=0x7f91e0008040 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 msgr2=0x7f91f0155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91e800d770 tx=0x7f91e800dc30 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 shutdown_connections 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f01f53d0 0x7f91f0155f90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91f01f3830 0x7f91f01f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 --2- 172.21.15.94:0/3516537085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91d40040d0 0x7f91f01e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 >> 172.21.15.94:0/3516537085 conn(0x7f91f00b3830 msgr2=0x7f91f0154910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 shutdown_connections 2024-05-27T12:35:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.687+0000 7f91f6000640 1 -- 172.21.15.94:0/3516537085 wait complete. 2024-05-27T12:35:52.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:52.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.891+0000 7f126be00640 1 Processor -- start 2024-05-27T12:35:52.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.891+0000 7f126be00640 1 -- start start 2024-05-27T12:35:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f126c145cf0 0x7f126c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c1460d0 0x7f126c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f126c0ce610 0x7f126c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f126c0ceae0 con 0x7f126c0ce610 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f126c1602c0 con 0x7f126c145cf0 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.892+0000 7f126be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f126c1605a0 con 0x7f126c1460d0 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f126c0ce610 0x7f126c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f126c0ce610 0x7f126c071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38690/0 (socket says 172.21.15.94:38690) 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 -- 172.21.15.94:0/1018255438 learned_addr learned my addr 172.21.15.94:0/1018255438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126aa00640 1 --2- 172.21.15.94:0/1018255438 >> v2:172.21.15.97:3300/0 conn(0x7f126c145cf0 0x7f126c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f1262000640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c1460d0 0x7f126c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 -- 172.21.15.94:0/1018255438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c1460d0 msgr2=0x7f126c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c1460d0 0x7f126c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 -- 172.21.15.94:0/1018255438 >> v2:172.21.15.97:3300/0 conn(0x7f126c145cf0 msgr2=0x7f126c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 --2- 172.21.15.94:0/1018255438 >> v2:172.21.15.97:3300/0 conn(0x7f126c145cf0 0x7f126c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.893+0000 7f126b400640 1 -- 172.21.15.94:0/1018255438 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c160880 con 0x7f126c0ce610 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.894+0000 7f126b400640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c0ce610 0x7f126c071d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f126c0b5790 tx=0x7f126402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2311805797bc8710 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126a000640 1 -- 172.21.15.94:0/1018255438 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036070 con 0x7f126c0ce610 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126a000640 1 -- 172.21.15.94:0/1018255438 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264031600 con 0x7f126c0ce610 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126a000640 1 -- 172.21.15.94:0/1018255438 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640319d0 con 0x7f126c0ce610 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 -- 172.21.15.94:0/1018255438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c0ce610 msgr2=0x7f126c071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c0ce610 0x7f126c071d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f126c0b5790 tx=0x7f126402a6a0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 -- 172.21.15.94:0/1018255438 shutdown_connections 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c0ce610 0x7f126c071d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 --2- 172.21.15.94:0/1018255438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c1460d0 0x7f126c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 --2- 172.21.15.94:0/1018255438 >> v2:172.21.15.97:3300/0 conn(0x7f126c145cf0 0x7f126c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 -- 172.21.15.94:0/1018255438 >> 172.21.15.94:0/1018255438 conn(0x7f126c0b2160 msgr2=0x7f126c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 -- 172.21.15.94:0/1018255438 shutdown_connections 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.895+0000 7f126be00640 1 -- 172.21.15.94:0/1018255438 wait complete. 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.897+0000 7f126be00640 1 Processor -- start 2024-05-27T12:35:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.897+0000 7f126be00640 1 -- start start 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c0ce610 0x7f126c0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 0x7f126c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f126c06ec80 con 0x7f126c1460d0 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f126c06eb00 con 0x7f126c0ce610 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f126c0722a0 con 0x7f126c145cf0 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38704/0 (socket says 172.21.15.94:38704) 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.898+0000 7f126b400640 1 -- 172.21.15.94:0/1564141555 learned_addr learned my addr 172.21.15.94:0/1564141555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f1262000640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 0x7f126c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126aa00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c0ce610 0x7f126c0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 -- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 msgr2=0x7f126c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 0x7f126c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 -- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c0ce610 msgr2=0x7f126c0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c0ce610 0x7f126c0c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 -- 172.21.15.94:0/1564141555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c0c6630 con 0x7f126c1460d0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f1262000640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 0x7f126c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126b400640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f126402ac90 tx=0x7f1264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126403f020 con 0x7f126c1460d0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1264037040 con 0x7f126c1460d0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c0c6910 con 0x7f126c1460d0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.899+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264036070 con 0x7f126c1460d0 2024-05-27T12:35:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.900+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c0c6ed0 con 0x7f126c1460d0 2024-05-27T12:35:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.901+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c075460 con 0x7f126c1460d0 2024-05-27T12:35:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.902+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1264042020 con 0x7f126c1460d0 2024-05-27T12:35:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.903+0000 7f1263e00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 0x7f124806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.903+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f12640ac130 con 0x7f126c1460d0 2024-05-27T12:35:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.904+0000 7f126aa00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 0x7f124806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.905+0000 7f126aa00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 0x7f124806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f125c006fd0 tx=0x7f125c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:52.907+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1264049030 con 0x7f126c1460d0 2024-05-27T12:35:53.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.091+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f126c072e00 con 0x7f126c1460d0 2024-05-27T12:35:53.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.092+0000 7f1263e00640 1 -- 172.21.15.94:0/1564141555 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7f126407a440 con 0x7f126c1460d0 2024-05-27T12:35:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:53.092 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:46.194696+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 msgr2=0x7f124806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 0x7f124806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f125c006fd0 tx=0x7f125c008040 comp rx=0 tx=0).stop 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 msgr2=0x7f126c142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f126402ac90 tx=0x7f1264004a40 comp rx=0 tx=0).stop 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 shutdown_connections 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f124806c920 0x7f124806ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1460d0 0x7f126c142860 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f126c145cf0 0x7f126c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 --2- 172.21.15.94:0/1564141555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c0ce610 0x7f126c0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.094+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 >> 172.21.15.94:0/1564141555 conn(0x7f126c0b2160 msgr2=0x7f126c0a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.095+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 shutdown_connections 2024-05-27T12:35:53.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.095+0000 7f126be00640 1 -- 172.21.15.94:0/1564141555 wait complete. 2024-05-27T12:35:53.112 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-27T12:35:53.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.310+0000 7f1540e00640 1 Processor -- start 2024-05-27T12:35:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.310+0000 7f1540e00640 1 -- start start 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1540e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1540e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f153c072fb0 0x7f153c071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1540e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f153c06dfc0 con 0x7f153c072460 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1540e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f153c0728b0 con 0x7f153c072fb0 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1540e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f153c06ed10 con 0x7f153c149160 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1539400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f153c072fb0 0x7f153c071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1539400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f153c072fb0 0x7f153c071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38718/0 (socket says 172.21.15.94:38718) 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.312+0000 7f1539400640 1 -- 172.21.15.94:0/2616939327 learned_addr learned my addr 172.21.15.94:0/2616939327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539e00640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539400640 1 -- 172.21.15.94:0/2616939327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 msgr2=0x7f153c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539400640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539400640 1 -- 172.21.15.94:0/2616939327 >> v1:172.21.15.97:6789/0 conn(0x7f153c072460 legacy=0x7f153c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539400640 1 -- 172.21.15.94:0/2616939327 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c149540 con 0x7f153c072fb0 2024-05-27T12:35:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.313+0000 7f1539e00640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.314+0000 7f1539400640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072fb0 0x7f153c071f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f883fc4f62c2e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.314+0000 7f1538a00640 1 -- 172.21.15.94:0/2616939327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528037070 con 0x7f153c072fb0 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.314+0000 7f1538a00640 1 -- 172.21.15.94:0/2616939327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1528032440 con 0x7f153c072fb0 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.314+0000 7f1538a00640 1 -- 172.21.15.94:0/2616939327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528032810 con 0x7f153c072fb0 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 -- 172.21.15.94:0/2616939327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072fb0 msgr2=0x7f153c071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072fb0 0x7f153c071f20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152802a210 comp rx=0 tx=0).stop 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 -- 172.21.15.94:0/2616939327 shutdown_connections 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072fb0 0x7f153c071f20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 --2- 172.21.15.94:0/2616939327 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 -- 172.21.15.94:0/2616939327 >> 172.21.15.94:0/2616939327 conn(0x7f153c0b3830 msgr2=0x7f153c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 -- 172.21.15.94:0/2616939327 shutdown_connections 2024-05-27T12:35:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.315+0000 7f1540e00640 1 -- 172.21.15.94:0/2616939327 wait complete. 2024-05-27T12:35:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.317+0000 7f1540e00640 1 Processor -- start 2024-05-27T12:35:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.317+0000 7f1540e00640 1 -- start start 2024-05-27T12:35:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f153c0c8130 0x7f153c1ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f153c0d03a0 con 0x7f153c072460 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f153c06e510 con 0x7f153c0c8130 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1540e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f153c0cff30 con 0x7f153c149160 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38732/0 (socket says 172.21.15.94:38732) 2024-05-27T12:35:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.318+0000 7f1539e00640 1 -- 172.21.15.94:0/263602831 learned_addr learned my addr 172.21.15.94:0/263602831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 -- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 msgr2=0x7f153c0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 -- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f153c0c8130 msgr2=0x7f153c1ed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f153c0c8130 0x7f153c1ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 -- 172.21.15.94:0/263602831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c0c58b0 con 0x7f153c072460 2024-05-27T12:35:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1539e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c7f0 tx=0x7f152c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c015070 con 0x7f153c072460 2024-05-27T12:35:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152c007a30 con 0x7f153c072460 2024-05-27T12:35:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c007e00 con 0x7f153c072460 2024-05-27T12:35:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c0c5b90 con 0x7f153c072460 2024-05-27T12:35:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.319+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1629b0 con 0x7f153c072460 2024-05-27T12:35:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.320+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c0c8550 con 0x7f153c072460 2024-05-27T12:35:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.321+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f152c018070 con 0x7f153c072460 2024-05-27T12:35:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.322+0000 7f1532000640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.322+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f152c08bbf0 con 0x7f153c072460 2024-05-27T12:35:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.323+0000 7f1539400640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.324+0000 7f1539400640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 0x7f151c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1528034000 tx=0x7f152802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.326+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f152c059f90 con 0x7f153c072460 2024-05-27T12:35:53.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.566+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f153c075460 con 0x7f153c072460 2024-05-27T12:35:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.568+0000 7f1532000640 1 -- 172.21.15.94:0/263602831 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f152c059890 con 0x7f153c072460 2024-05-27T12:35:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:53.567 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 msgr2=0x7f151c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 0x7f151c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1528034000 tx=0x7f152802c000 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 msgr2=0x7f153c162470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c7f0 tx=0x7f152c00cbf0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 shutdown_connections 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f153c0c8130 0x7f153c1ed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f153c149160 0x7f153c0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 --2- 172.21.15.94:0/263602831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f153c072460 0x7f153c162470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 >> 172.21.15.94:0/263602831 conn(0x7f153c0b3830 msgr2=0x7f153c14b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 shutdown_connections 2024-05-27T12:35:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.571+0000 7f1540e00640 1 -- 172.21.15.94:0/263602831 wait complete. 2024-05-27T12:35:53.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:35:53.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.787+0000 7f59d9000640 1 Processor -- start 2024-05-27T12:35:53.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.787+0000 7f59d9000640 1 -- start start 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.788+0000 7f59d9000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f59d406f5e0 0x7f59d4071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.788+0000 7f59d9000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 0x7f59d40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59d9000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d413e9e0 con 0x7f59d413e600 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59d9000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f59d40cde50 con 0x7f59d406f5e0 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59d9000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f59d415ba50 con 0x7f59d40cd480 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f59d406f5e0 0x7f59d4071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f59d406f5e0 0x7f59d4071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38746/0 (socket says 172.21.15.94:38746) 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 -- 172.21.15.94:0/2003800547 learned_addr learned my addr 172.21.15.94:0/2003800547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59d3e00640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 0x7f59d40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 -- 172.21.15.94:0/2003800547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 msgr2=0x7f59d40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 0x7f59d40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 -- 172.21.15.94:0/2003800547 >> v1:172.21.15.97:6789/0 conn(0x7f59d413e600 legacy=0x7f59d406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:53.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59cbe00640 1 -- 172.21.15.94:0/2003800547 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d415bd10 con 0x7f59d406f5e0 2024-05-27T12:35:53.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.789+0000 7f59d3e00640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 0x7f59d40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.790+0000 7f59cbe00640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4071d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009870 tx=0x7f59bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5025a2df6d3acbea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.791+0000 7f59d2a00640 1 -- 172.21.15.94:0/2003800547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc037070 con 0x7f59d406f5e0 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.791+0000 7f59d2a00640 1 -- 172.21.15.94:0/2003800547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc032440 con 0x7f59d406f5e0 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.791+0000 7f59d2a00640 1 -- 172.21.15.94:0/2003800547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc032810 con 0x7f59d406f5e0 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.791+0000 7f59d9000640 1 -- 172.21.15.94:0/2003800547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 msgr2=0x7f59d4071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.791+0000 7f59d9000640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4071d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009870 tx=0x7f59bc02a210 comp rx=0 tx=0).stop 2024-05-27T12:35:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 -- 172.21.15.94:0/2003800547 shutdown_connections 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d40cd480 0x7f59d40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 --2- 172.21.15.94:0/2003800547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4071d60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 -- 172.21.15.94:0/2003800547 >> 172.21.15.94:0/2003800547 conn(0x7f59d40b2160 msgr2=0x7f59d40b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 -- 172.21.15.94:0/2003800547 shutdown_connections 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.792+0000 7f59d9000640 1 -- 172.21.15.94:0/2003800547 wait complete. 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.793+0000 7f59d9000640 1 Processor -- start 2024-05-27T12:35:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.793+0000 7f59d9000640 1 -- start start 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 0x7f59d4156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d413e600 0x7f59d413a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59d415c5b0 con 0x7f59d406f5e0 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59d415c2d0 con 0x7f59d40cd480 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.794+0000 7f59d9000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f59d415c920 con 0x7f59d413e600 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38758/0 (socket says 172.21.15.94:38758) 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 -- 172.21.15.94:0/3524625330 learned_addr learned my addr 172.21.15.94:0/3524625330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3e00640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d413e600 0x7f59d413a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 -- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d413e600 msgr2=0x7f59d413a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59cbe00640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 0x7f59d4156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d413e600 0x7f59d413a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 -- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 msgr2=0x7f59d4156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 0x7f59d4156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 -- 172.21.15.94:0/3524625330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d4155ff0 con 0x7f59d406f5e0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59cbe00640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 0x7f59d4156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.795+0000 7f59d3400640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c460 tx=0x7f59c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.796+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4016020 con 0x7f59d406f5e0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.796+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4012870 con 0x7f59d406f5e0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.796+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d413ad60 con 0x7f59d406f5e0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.796+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4012c60 con 0x7f59d406f5e0 2024-05-27T12:35:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.796+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d413b320 con 0x7f59d406f5e0 2024-05-27T12:35:53.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.797+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d4075460 con 0x7f59d406f5e0 2024-05-27T12:35:53.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.798+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59c4019020 con 0x7f59d406f5e0 2024-05-27T12:35:53.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.799+0000 7f59cb400640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 0x7f59b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:53.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.799+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f59c408b6f0 con 0x7f59d406f5e0 2024-05-27T12:35:53.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.799+0000 7f59cbe00640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 0x7f59b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:53.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.800+0000 7f59cbe00640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 0x7f59b006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59bc034000 tx=0x7f59bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:53.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:53.803+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59c4020070 con 0x7f59d406f5e0 2024-05-27T12:35:54.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.038+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59d4140c20 con 0x7f59d406f5e0 2024-05-27T12:35:54.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.039+0000 7f59cb400640 1 -- 172.21.15.94:0/3524625330 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f59c4059a00 con 0x7f59d406f5e0 2024-05-27T12:35:54.039 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:54.039 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/1416034903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1416034903},{"type":"v1","addr":"172.21.15.94:6833","nonce":1416034903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2839268807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2839268807},{"type":"v1","addr":"172.21.15.94:6835","nonce":2839268807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1712034361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1712034361},{"type":"v1","addr":"172.21.15.97:6837","nonce":1712034361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:34:59.347690+0000","modified":"2024-05-27T12:35:02.434158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6836/1775076322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1775076322},{"type":"v1","addr":"172.21.15.97:6836","nonce":1775076322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4244,"qdb_cluster":[4244]},"id":1}]} 2024-05-27T12:35:54.039 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-05-27T12:35:54.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 msgr2=0x7f59b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 0x7f59b006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59bc034000 tx=0x7f59bc02e000 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 msgr2=0x7f59d4155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c460 tx=0x7f59c400c920 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 shutdown_connections 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f59b006c920 0x7f59b006ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d40cd480 0x7f59d4156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59d413e600 0x7f59d413a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 --2- 172.21.15.94:0/3524625330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59d406f5e0 0x7f59d4155ab0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 >> 172.21.15.94:0/3524625330 conn(0x7f59d40b2160 msgr2=0x7f59d415d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 shutdown_connections 2024-05-27T12:35:54.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.042+0000 7f59d9000640 1 -- 172.21.15.94:0/3524625330 wait complete. 2024-05-27T12:35:54.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:35:54.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.247+0000 7fb3ccc00640 1 Processor -- start 2024-05-27T12:35:54.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.247+0000 7fb3ccc00640 1 -- start start 2024-05-27T12:35:54.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3ccc00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3c80b79c0 0x7fb3c806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3ccc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3c806d3c0 0x7fb3c80d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3ccc00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c806d7d0 con 0x7fb3c81620b0 2024-05-27T12:35:54.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3ccc00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80d56d0 con 0x7fb3c80b79c0 2024-05-27T12:35:54.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3ccc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80cffe0 con 0x7fb3c806d3c0 2024-05-27T12:35:54.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3c5e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb3c81620b0 0x7fb3c80cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54074/0 (socket says 172.21.15.94:54074) 2024-05-27T12:35:54.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3c5e00640 1 -- 172.21.15.94:0/3428994772 learned_addr learned my addr 172.21.15.94:0/3428994772 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:54.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.249+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3428994772 >> v2:172.21.15.94:3300/0 conn(0x7fb3c80b79c0 0x7fb3c806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c6800640 1 --2- 172.21.15.94:0/3428994772 >> v2:172.21.15.97:3300/0 conn(0x7fb3c806d3c0 0x7fb3c80d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c5400640 1 -- 172.21.15.94:0/3428994772 >> v2:172.21.15.97:3300/0 conn(0x7fb3c806d3c0 msgr2=0x7fb3c80d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3428994772 >> v2:172.21.15.97:3300/0 conn(0x7fb3c806d3c0 0x7fb3c80d3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c4a00640 1 -- 172.21.15.94:0/3428994772 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679904860 0 0) 0x7fb3b8001110 con 0x7fb3c81620b0 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c5400640 1 -- 172.21.15.94:0/3428994772 >> v1:172.21.15.94:6789/0 conn(0x7fb3c81620b0 legacy=0x7fb3c80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.250+0000 7fb3c5400640 1 -- 172.21.15.94:0/3428994772 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80b39a0 con 0x7fb3c80b79c0 2024-05-27T12:35:54.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.251+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3428994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c806ce80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b7770 tx=0x7fb3b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dba2d635d2342c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.251+0000 7fb3c4a00640 1 -- 172.21.15.94:0/3428994772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4037070 con 0x7fb3c80b79c0 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.251+0000 7fb3c4a00640 1 -- 172.21.15.94:0/3428994772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b4032440 con 0x7fb3c80b79c0 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.251+0000 7fb3c4a00640 1 -- 172.21.15.94:0/3428994772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4032810 con 0x7fb3c80b79c0 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3428994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 msgr2=0x7fb3c806ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3428994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c806ce80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b7770 tx=0x7fb3b402a1a0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3428994772 shutdown_connections 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3428994772 >> v2:172.21.15.97:3300/0 conn(0x7fb3c806d3c0 0x7fb3c80d3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3428994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c806ce80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3428994772 >> 172.21.15.94:0/3428994772 conn(0x7fb3c80d1940 msgr2=0x7fb3c80d4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3428994772 shutdown_connections 2024-05-27T12:35:54.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.252+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3428994772 wait complete. 2024-05-27T12:35:54.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.254+0000 7fb3ccc00640 1 Processor -- start 2024-05-27T12:35:54.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.254+0000 7fb3ccc00640 1 -- start start 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c806d3c0 0x7fb3c815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3c81620b0 0x7fb3c80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80d3150 con 0x7fb3c80b79c0 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80d2e70 con 0x7fb3c806d3c0 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3ccc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c80d3430 con 0x7fb3c81620b0 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38778/0 (socket says 172.21.15.94:38778) 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 -- 172.21.15.94:0/3129532062 learned_addr learned my addr 172.21.15.94:0/3129532062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:54.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c6800640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3c81620b0 0x7fb3c80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 -- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3c81620b0 msgr2=0x7fb3c80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5e00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c806d3c0 0x7fb3c815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3c81620b0 0x7fb3c80c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 -- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c806d3c0 msgr2=0x7fb3c815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c806d3c0 0x7fb3c815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.255+0000 7fb3c5400640 1 -- 172.21.15.94:0/3129532062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80c93d0 con 0x7fb3c80b79c0 2024-05-27T12:35:54.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.256+0000 7fb3c5400640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4000c00 tx=0x7fb3b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.258+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4037050 con 0x7fb3c80b79c0 2024-05-27T12:35:54.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.258+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b4038040 con 0x7fb3c80b79c0 2024-05-27T12:35:54.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.258+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004450 con 0x7fb3c80b79c0 2024-05-27T12:35:54.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.258+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c80c9670 con 0x7fb3c80b79c0 2024-05-27T12:35:54.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.258+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80c9c30 con 0x7fb3c80b79c0 2024-05-27T12:35:54.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.260+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80ba800 con 0x7fb3c80b79c0 2024-05-27T12:35:54.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.260+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3b4030980 con 0x7fb3c80b79c0 2024-05-27T12:35:54.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.261+0000 7fb3be000640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 0x7fb3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.261+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb3b40ac820 con 0x7fb3c80b79c0 2024-05-27T12:35:54.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.262+0000 7fb3c5e00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 0x7fb3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.263+0000 7fb3c5e00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 0x7fb3a806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8000f60 tx=0x7fb3b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.266+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3c80ba800 con 0x7fb3c80b79c0 2024-05-27T12:35:54.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.467+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3c8075460 con 0x7fb3c80b79c0 2024-05-27T12:35:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.470+0000 7fb3be000640 1 -- 172.21.15.94:0/3129532062 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7fb3b407abd0 con 0x7fb3c80b79c0 2024-05-27T12:35:54.470 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:54.470 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:46.194696+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:34:56.888194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T12:34:58.330763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:35:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 msgr2=0x7fb3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 0x7fb3a806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8000f60 tx=0x7fb3b8009040 comp rx=0 tx=0).stop 2024-05-27T12:35:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 msgr2=0x7fb3c8150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4000c00 tx=0x7fb3b40046c0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 shutdown_connections 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb3a806c8f0 0x7fb3a806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3c81620b0 0x7fb3c80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c80b79c0 0x7fb3c8150cf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 --2- 172.21.15.94:0/3129532062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3c806d3c0 0x7fb3c815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 >> 172.21.15.94:0/3129532062 conn(0x7fb3c80d1940 msgr2=0x7fb3c80cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 shutdown_connections 2024-05-27T12:35:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.473+0000 7fb3ccc00640 1 -- 172.21.15.94:0/3129532062 wait complete. 2024-05-27T12:35:54.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-27T12:35:54.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.689+0000 7f7dc3a00640 1 Processor -- start 2024-05-27T12:35:54.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.690+0000 7f7dc3a00640 1 -- start start 2024-05-27T12:35:54.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc162150 0x7f7dbc0d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7dbc06ca40 0x7f7dbc09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7dbc09af70 0x7f7dbc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc09b3c0 con 0x7f7dbc09af70 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc0d2020 con 0x7f7dbc06ca40 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc3a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc0d1d40 con 0x7f7dbc162150 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc1a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7dbc09af70 0x7f7dbc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc1a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7dbc09af70 0x7f7dbc06d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38794/0 (socket says 172.21.15.94:38794) 2024-05-27T12:35:54.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.691+0000 7f7dc1a00640 1 -- 172.21.15.94:0/1185575309 learned_addr learned my addr 172.21.15.94:0/1185575309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1000640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc162150 0x7f7dbc0d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dbbe00640 1 --2- 172.21.15.94:0/1185575309 >> v2:172.21.15.97:3300/0 conn(0x7f7dbc06ca40 0x7f7dbc09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1a00640 1 -- 172.21.15.94:0/1185575309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc162150 msgr2=0x7f7dbc0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1a00640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc162150 0x7f7dbc0d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1a00640 1 -- 172.21.15.94:0/1185575309 >> v2:172.21.15.97:3300/0 conn(0x7f7dbc06ca40 msgr2=0x7f7dbc09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1a00640 1 --2- 172.21.15.94:0/1185575309 >> v2:172.21.15.97:3300/0 conn(0x7f7dbc06ca40 0x7f7dbc09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.692+0000 7f7dc1a00640 1 -- 172.21.15.94:0/1185575309 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc0ccd60 con 0x7f7dbc09af70 2024-05-27T12:35:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.693+0000 7f7dc1a00640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc09af70 0x7f7dbc06d100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0738b0 tx=0x7f7db402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea224a40b9ec3a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.693+0000 7f7dbb400640 1 -- 172.21.15.94:0/1185575309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4037070 con 0x7f7dbc09af70 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.693+0000 7f7dbb400640 1 -- 172.21.15.94:0/1185575309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db4032440 con 0x7f7dbc09af70 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.693+0000 7f7dbb400640 1 -- 172.21.15.94:0/1185575309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4032810 con 0x7f7dbc09af70 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 -- 172.21.15.94:0/1185575309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc09af70 msgr2=0x7f7dbc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc09af70 0x7f7dbc06d100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0738b0 tx=0x7f7db402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 -- 172.21.15.94:0/1185575309 shutdown_connections 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc09af70 0x7f7dbc06d100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/1185575309 >> v2:172.21.15.97:3300/0 conn(0x7f7dbc06ca40 0x7f7dbc09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/1185575309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc162150 0x7f7dbc0d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 -- 172.21.15.94:0/1185575309 >> 172.21.15.94:0/1185575309 conn(0x7f7dbc0b4160 msgr2=0x7f7dbc0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 -- 172.21.15.94:0/1185575309 shutdown_connections 2024-05-27T12:35:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.694+0000 7f7dc3a00640 1 -- 172.21.15.94:0/1185575309 wait complete. 2024-05-27T12:35:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.696+0000 7f7dc3a00640 1 Processor -- start 2024-05-27T12:35:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.696+0000 7f7dc3a00640 1 -- start start 2024-05-27T12:35:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc09af70 0x7f7dbc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dbc162150 0x7f7dbc15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0d1a30 con 0x7f7dbc06ca40 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0cd320 con 0x7f7dbc162150 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc3a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc06d640 con 0x7f7dbc09af70 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38804/0 (socket says 172.21.15.94:38804) 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 -- 172.21.15.94:0/2167543817 learned_addr learned my addr 172.21.15.94:0/2167543817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dbbe00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc09af70 0x7f7dbc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dbc162150 0x7f7dbc15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 -- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc09af70 msgr2=0x7f7dbc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc09af70 0x7f7dbc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 -- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dbc162150 msgr2=0x7f7dbc15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dbc162150 0x7f7dbc15dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.697+0000 7f7dc1000640 1 -- 172.21.15.94:0/2167543817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc0c9950 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7dc1000640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c7f0 tx=0x7f7da400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4015070 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da4004220 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc0c9c30 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4007620 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.698+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc152d90 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.699+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc068a70 con 0x7f7dbc06ca40 2024-05-27T12:35:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.700+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7da40079d0 con 0x7f7dbc06ca40 2024-05-27T12:35:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.701+0000 7f7db8c00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 0x7f7d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.701+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7da408bcd0 con 0x7f7dbc06ca40 2024-05-27T12:35:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.702+0000 7f7dbbe00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 0x7f7d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.703+0000 7f7dbbe00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 0x7f7d9806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc15c040 tx=0x7f7dac007560 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:54.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.706+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7da4059fe0 con 0x7f7dbc06ca40 2024-05-27T12:35:54.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.939+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7dbc0cfbe0 con 0x7f7dbc06ca40 2024-05-27T12:35:54.940 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-05-27T12:35:54.940+0000 7fd94c600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T12:35:54.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.944+0000 7f7db8c00640 1 -- 172.21.15.94:0/2167543817 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f7da405ee20 con 0x7f7dbc06ca40 2024-05-27T12:35:54.943 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-27T12:35:54.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 msgr2=0x7f7d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 0x7f7d9806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc15c040 tx=0x7f7dac007560 comp rx=0 tx=0).stop 2024-05-27T12:35:54.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 msgr2=0x7f7dbc152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:54.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c7f0 tx=0x7f7da400ccb0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 shutdown_connections 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d9806c8f0 0x7f7d9806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dbc162150 0x7f7dbc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dbc09af70 0x7f7dbc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 --2- 172.21.15.94:0/2167543817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dbc06ca40 0x7f7dbc152850 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 >> 172.21.15.94:0/2167543817 conn(0x7f7dbc0b4160 msgr2=0x7f7dbc0cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 shutdown_connections 2024-05-27T12:35:54.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:54.947+0000 7f7dc3a00640 1 -- 172.21.15.94:0/2167543817 wait complete. 2024-05-27T12:35:54.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-27T12:35:55.062 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2024-05-27T12:35:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.163+0000 7fe773200640 1 Processor -- start 2024-05-27T12:35:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.163+0000 7fe773200640 1 -- start start 2024-05-27T12:35:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe773200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe77415e790 0x7fe7740ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe773200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe773200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe77406cd60 con 0x7fe7740aef90 2024-05-27T12:35:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe773200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe77406d040 con 0x7fe77415e790 2024-05-27T12:35:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe773200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe7740b4760 con 0x7fe7740b5f70 2024-05-27T12:35:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.165+0000 7fe771e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe77415e790 0x7fe7740ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe77415e790 0x7fe7740ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:48016/0 (socket says 172.21.15.94:48016) 2024-05-27T12:35:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 -- 172.21.15.94:0/3451729461 learned_addr learned my addr 172.21.15.94:0/3451729461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe772800640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 -- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 msgr2=0x7fe7740ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 -- 172.21.15.94:0/3451729461 >> v1:172.21.15.97:6789/0 conn(0x7fe7740aef90 legacy=0x7fe7741623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.166+0000 7fe771e00640 1 -- 172.21.15.94:0/3451729461 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7740b4a40 con 0x7fe77415e790 2024-05-27T12:35:55.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.168+0000 7fe771e00640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe77415e790 0x7fe7740ae2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240914cc10450f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:55.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.168+0000 7fe771400640 1 -- 172.21.15.94:0/3451729461 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764037070 con 0x7fe77415e790 2024-05-27T12:35:55.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.168+0000 7fe771400640 1 -- 172.21.15.94:0/3451729461 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764032440 con 0x7fe77415e790 2024-05-27T12:35:55.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.168+0000 7fe771400640 1 -- 172.21.15.94:0/3451729461 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764032810 con 0x7fe77415e790 2024-05-27T12:35:55.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 -- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe77415e790 msgr2=0x7fe7740ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe77415e790 0x7fe7740ae2a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76402a250 comp rx=0 tx=0).stop 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 -- 172.21.15.94:0/3451729461 shutdown_connections 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 --2- 172.21.15.94:0/3451729461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe77415e790 0x7fe7740ae2a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 -- 172.21.15.94:0/3451729461 >> 172.21.15.94:0/3451729461 conn(0x7fe7740d0e50 msgr2=0x7fe7740d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 -- 172.21.15.94:0/3451729461 shutdown_connections 2024-05-27T12:35:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.169+0000 7fe773200640 1 -- 172.21.15.94:0/3451729461 wait complete. 2024-05-27T12:35:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.171+0000 7fe773200640 1 Processor -- start 2024-05-27T12:35:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.171+0000 7fe773200640 1 -- start start 2024-05-27T12:35:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7740aef90 0x7fe7741439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406c990 con 0x7fe77415e790 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406c810 con 0x7fe7740aef90 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe773200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe77406cb10 con 0x7fe7740b5f70 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe772800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe772800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38820/0 (socket says 172.21.15.94:38820) 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe772800640 1 -- 172.21.15.94:0/2667555680 learned_addr learned my addr 172.21.15.94:0/2667555680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.172+0000 7fe771e00640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7740aef90 0x7fe7741439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe769400640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 -- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 msgr2=0x7fe7740cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 -- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7740aef90 msgr2=0x7fe7741439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7740aef90 0x7fe7741439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 -- 172.21.15.94:0/2667555680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe774143ef0 con 0x7fe77415e790 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe769400640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe772800640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00efc0 tx=0x7fe76c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c018020 con 0x7fe77415e790 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76c014480 con 0x7fe77415e790 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c014870 con 0x7fe77415e790 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7740cd750 con 0x7fe77415e790 2024-05-27T12:35:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.173+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe774144710 con 0x7fe77415e790 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.175+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe774075460 con 0x7fe77415e790 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.176+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe76c005d10 con 0x7fe77415e790 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.177+0000 7fe76aa00640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 0x7fe75006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.177+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fe76c08cbb0 con 0x7fe77415e790 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.178+0000 7fe771e00640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 0x7fe75006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.179+0000 7fe771e00640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 0x7fe75006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe764034000 tx=0x7fe76402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.182+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe76c022030 con 0x7fe77415e790 2024-05-27T12:35:55.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.424+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe7740d06e0 con 0x7fe77415e790 2024-05-27T12:35:55.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.948+0000 7fe76aa00640 1 -- 172.21.15.94:0/2667555680 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fe76c010040 con 0x7fe77415e790 2024-05-27T12:35:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.952+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 msgr2=0x7fe75006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 0x7fe75006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe764034000 tx=0x7fe76402e000 comp rx=0 tx=0).stop 2024-05-27T12:35:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 msgr2=0x7fe77406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00efc0 tx=0x7fe76c00c490 comp rx=0 tx=0).stop 2024-05-27T12:35:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 shutdown_connections 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe75006c8f0 0x7fe75006edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe77415e790 0x7fe77406ee20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7740b5f70 0x7fe7740cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 --2- 172.21.15.94:0/2667555680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7740aef90 0x7fe7741439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 >> 172.21.15.94:0/2667555680 conn(0x7fe7740d0e50 msgr2=0x7fe7740b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.953+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 shutdown_connections 2024-05-27T12:35:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:55.954+0000 7fe773200640 1 -- 172.21.15.94:0/2667555680 wait complete. 2024-05-27T12:35:55.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-27T12:35:56.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.174+0000 7f1344a00640 1 Processor -- start 2024-05-27T12:35:56.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.175+0000 7f1344a00640 1 -- start start 2024-05-27T12:35:56.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.176+0000 7f1344a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1340144e70 0x7f13400cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.176+0000 7f1344a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13400b4670 0x7f13400b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.176+0000 7f1344a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13400cdd80 con 0x7f13400cfd20 2024-05-27T12:35:56.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.176+0000 7f1344a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f13400b5020 con 0x7f1340144e70 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.176+0000 7f1344a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1340145250 con 0x7f13400b4670 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1340144e70 0x7f13400cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1340144e70 0x7f13400cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38836/0 (socket says 172.21.15.94:38836) 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 -- 172.21.15.94:0/664049153 learned_addr learned my addr 172.21.15.94:0/664049153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133f200640 1 --2- 172.21.15.94:0/664049153 >> v2:172.21.15.97:3300/0 conn(0x7f13400b4670 0x7f13400b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 -- 172.21.15.94:0/664049153 >> v2:172.21.15.97:3300/0 conn(0x7f13400b4670 msgr2=0x7f13400b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 --2- 172.21.15.94:0/664049153 >> v2:172.21.15.97:3300/0 conn(0x7f13400b4670 0x7f13400b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 -- 172.21.15.94:0/664049153 >> v1:172.21.15.97:6789/0 conn(0x7f13400cfd20 legacy=0x7f134015f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:56.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.177+0000 7f133e800640 1 -- 172.21.15.94:0/664049153 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13400d01a0 con 0x7f1340144e70 2024-05-27T12:35:56.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.178+0000 7f133e800640 1 --2- 172.21.15.94:0/664049153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f13400cd0c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1340070750 tx=0x7f133002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1956156bdfd8d0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:56.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.178+0000 7f133de00640 1 -- 172.21.15.94:0/664049153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330037070 con 0x7f1340144e70 2024-05-27T12:35:56.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.178+0000 7f133de00640 1 -- 172.21.15.94:0/664049153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1330032440 con 0x7f1340144e70 2024-05-27T12:35:56.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.178+0000 7f133de00640 1 -- 172.21.15.94:0/664049153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330032810 con 0x7f1340144e70 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 -- 172.21.15.94:0/664049153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 msgr2=0x7f13400cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 --2- 172.21.15.94:0/664049153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f13400cd0c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1340070750 tx=0x7f133002a230 comp rx=0 tx=0).stop 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 -- 172.21.15.94:0/664049153 shutdown_connections 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 --2- 172.21.15.94:0/664049153 >> v2:172.21.15.97:3300/0 conn(0x7f13400b4670 0x7f13400b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 --2- 172.21.15.94:0/664049153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f13400cd0c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 -- 172.21.15.94:0/664049153 >> 172.21.15.94:0/664049153 conn(0x7f134006b930 msgr2=0x7f134006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 -- 172.21.15.94:0/664049153 shutdown_connections 2024-05-27T12:35:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.179+0000 7f1344a00640 1 -- 172.21.15.94:0/664049153 wait complete. 2024-05-27T12:35:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.181+0000 7f1344a00640 1 Processor -- start 2024-05-27T12:35:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.181+0000 7f1344a00640 1 -- start start 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 0x7f13400b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 0x7f134013b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13400cdaa0 con 0x7f1340144e70 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f134015f4c0 con 0x7f13400b4670 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.182+0000 7f1344a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f134015f150 con 0x7f13400cfd20 2024-05-27T12:35:56.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38842/0 (socket says 172.21.15.94:38842) 2024-05-27T12:35:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 -- 172.21.15.94:0/3543027391 learned_addr learned my addr 172.21.15.94:0/3543027391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133e800640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 0x7f13400b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f1337e00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 0x7f134013b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 -- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 msgr2=0x7f134013b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 0x7f134013b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 -- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 msgr2=0x7f13400b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 0x7f13400b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 -- 172.21.15.94:0/3543027391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134013c340 con 0x7f1340144e70 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f1337e00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 0x7f134013b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133e800640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 0x7f13400b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.183+0000 7f133f200640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f133800cd70 tx=0x7f133800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.184+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338003b90 con 0x7f1340144e70 2024-05-27T12:35:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.184+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338003d30 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.184+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338004240 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.184+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13401f5730 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.184+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401f5cf0 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.185+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13401f5ef0 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.186+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13380043e0 con 0x7f1340144e70 2024-05-27T12:35:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.187+0000 7f1336a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:56.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.187+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f133808c660 con 0x7f1340144e70 2024-05-27T12:35:56.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.188+0000 7f133e800640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:56.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.189+0000 7f133e800640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 0x7f131c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1330034000 tx=0x7f133002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:56.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.191+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13401f5ef0 con 0x7f1340144e70 2024-05-27T12:35:56.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:56.390+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1340075460 con 0x7f1340144e70 2024-05-27T12:35:57.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.284+0000 7f1336a00640 1 -- 172.21.15.94:0/3543027391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f133805aa50 con 0x7f1340144e70 2024-05-27T12:35:57.284 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 msgr2=0x7f131c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 0x7f131c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1330034000 tx=0x7f133002e000 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 msgr2=0x7f134013bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f133800cd70 tx=0x7f133800a660 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 shutdown_connections 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f131c06c8f0 0x7f131c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1340144e70 0x7f134013bd50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13400cfd20 0x7f134013b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 --2- 172.21.15.94:0/3543027391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13400b4670 0x7f13400b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 >> 172.21.15.94:0/3543027391 conn(0x7f134006b930 msgr2=0x7f134015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:57.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 shutdown_connections 2024-05-27T12:35:57.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.289+0000 7f1344a00640 1 -- 172.21.15.94:0/3543027391 wait complete. 2024-05-27T12:35:57.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-27T12:35:57.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.515+0000 7f2f6c800640 1 Processor -- start 2024-05-27T12:35:57.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.515+0000 7f2f6c800640 1 -- start start 2024-05-27T12:35:57.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f64146060 0x7f2f640b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f640cd8a0 0x7f2f6406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 0x7f2f6406be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6415dbe0 con 0x7f2f640cd8a0 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6415cfd0 con 0x7f2f640cdd00 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f6c800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6415d2b0 con 0x7f2f64146060 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f69400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f640cd8a0 0x7f2f6406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f69400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2f640cd8a0 0x7f2f6406b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38850/0 (socket says 172.21.15.94:38850) 2024-05-27T12:35:57.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.517+0000 7f2f69400640 1 -- 172.21.15.94:0/98872620 learned_addr learned my addr 172.21.15.94:0/98872620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:57.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f6a800640 1 --2- 172.21.15.94:0/98872620 >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 0x7f2f6406be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69e00640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f64146060 0x7f2f640b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69400640 1 -- 172.21.15.94:0/98872620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f64146060 msgr2=0x7f2f640b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69400640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f64146060 0x7f2f640b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69400640 1 -- 172.21.15.94:0/98872620 >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 msgr2=0x7f2f6406be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69400640 1 --2- 172.21.15.94:0/98872620 >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 0x7f2f6406be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f69400640 1 -- 172.21.15.94:0/98872620 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6415d590 con 0x7f2f640cd8a0 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.518+0000 7f2f6a800640 1 --2- 172.21.15.94:0/98872620 >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 0x7f2f6406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.519+0000 7f2f69400640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f640cd8a0 0x7f2f6406b930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f64071df0 tx=0x7f2f5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7998b4924e9b83 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.519+0000 7f2f68a00640 1 -- 172.21.15.94:0/98872620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54037070 con 0x7f2f640cd8a0 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.519+0000 7f2f68a00640 1 -- 172.21.15.94:0/98872620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54032440 con 0x7f2f640cd8a0 2024-05-27T12:35:57.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.519+0000 7f2f68a00640 1 -- 172.21.15.94:0/98872620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54032810 con 0x7f2f640cd8a0 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 -- 172.21.15.94:0/98872620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f640cd8a0 msgr2=0x7f2f6406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f640cd8a0 0x7f2f6406b930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2f64071df0 tx=0x7f2f5402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 -- 172.21.15.94:0/98872620 shutdown_connections 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 --2- 172.21.15.94:0/98872620 >> v2:172.21.15.97:3300/0 conn(0x7f2f640cdd00 0x7f2f6406be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f640cd8a0 0x7f2f6406b930 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 --2- 172.21.15.94:0/98872620 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f64146060 0x7f2f640b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 -- 172.21.15.94:0/98872620 >> 172.21.15.94:0/98872620 conn(0x7f2f640b2240 msgr2=0x7f2f640b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:57.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 -- 172.21.15.94:0/98872620 shutdown_connections 2024-05-27T12:35:57.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.520+0000 7f2f6c800640 1 -- 172.21.15.94:0/98872620 wait complete. 2024-05-27T12:35:57.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.522+0000 7f2f6c800640 1 Processor -- start 2024-05-27T12:35:57.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.522+0000 7f2f6c800640 1 -- start start 2024-05-27T12:35:57.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 0x7f2f64155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 0x7f2f641448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f640d0110 con 0x7f2f64146060 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64146440 con 0x7f2f640cdd00 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6c800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2f64072010 con 0x7f2f640cd8a0 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38862/0 (socket says 172.21.15.94:38862) 2024-05-27T12:35:57.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.523+0000 7f2f6a800640 1 -- 172.21.15.94:0/2915714048 learned_addr learned my addr 172.21.15.94:0/2915714048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:57.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f69400640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 0x7f2f641448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f69e00640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 0x7f2f64155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 -- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 msgr2=0x7f2f64155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 0x7f2f64155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 -- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 msgr2=0x7f2f641448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 0x7f2f641448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 -- 172.21.15.94:0/2915714048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f640adab0 con 0x7f2f64146060 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f69e00640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 0x7f2f64155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f69400640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 0x7f2f641448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f6a800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000cef0 tx=0x7f2f6000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6001a070 con 0x7f2f64146060 2024-05-27T12:35:57.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6000d840 con 0x7f2f64146060 2024-05-27T12:35:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.524+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000dc30 con 0x7f2f64146060 2024-05-27T12:35:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.525+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f640add90 con 0x7f2f64146060 2024-05-27T12:35:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.525+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f640ae870 con 0x7f2f64146060 2024-05-27T12:35:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.526+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f64068a70 con 0x7f2f64146060 2024-05-27T12:35:57.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.527+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f6001d070 con 0x7f2f64146060 2024-05-27T12:35:57.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.528+0000 7f2f5e000640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:57.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.528+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f2f6008c2d0 con 0x7f2f64146060 2024-05-27T12:35:57.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.529+0000 7f2f69e00640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:57.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.530+0000 7f2f69e00640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f58006fd0 tx=0x7f2f58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.532+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f6005a780 con 0x7f2f64146060 2024-05-27T12:35:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:57.731+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f640b3840 con 0x7f2f64146060 2024-05-27T12:35:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.288+0000 7f2f5e000640 1 -- 172.21.15.94:0/2915714048 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f2f6005a080 con 0x7f2f64146060 2024-05-27T12:35:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-05-27T12:35:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.291+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 msgr2=0x7f2f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.291+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 0x7f2f4806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f58006fd0 tx=0x7f2f58008040 comp rx=0 tx=0).stop 2024-05-27T12:35:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.291+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 msgr2=0x7f2f64144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.291+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000cef0 tx=0x7f2f6000a710 comp rx=0 tx=0).stop 2024-05-27T12:35:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 shutdown_connections 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2f4806c8f0 0x7f2f4806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f640cdd00 0x7f2f641448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f64146060 0x7f2f64144e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 --2- 172.21.15.94:0/2915714048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2f640cd8a0 0x7f2f64155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 >> 172.21.15.94:0/2915714048 conn(0x7f2f640b2240 msgr2=0x7f2f6406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 shutdown_connections 2024-05-27T12:35:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.292+0000 7f2f6c800640 1 -- 172.21.15.94:0/2915714048 wait complete. 2024-05-27T12:35:58.319 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T12:35:58.319 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-27T12:35:58.319 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-27T12:35:58.319 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T12:35:58.319 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T12:35:58.320 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-27T12:35:58.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-27T12:35:58.320 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T12:35:58.320 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-27T12:35:58.320 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-27T12:35:58.320 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-27T12:35:58.321 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T12:35:58.321 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-27T12:35:58.321 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-27T12:35:58.321 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-27T12:35:58.321 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T12:35:58.337 INFO:tasks.ceph.mds.c:Stopped 2024-05-27T12:35:58.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T12:35:58.341 INFO:tasks.ceph.mds.a:Stopped 2024-05-27T12:35:58.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T12:35:58.347 INFO:tasks.ceph.mds.d:Stopped 2024-05-27T12:35:58.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-27T12:35:58.348 INFO:tasks.ceph.mds.b:Stopped 2024-05-27T12:35:58.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-27T12:35:58.388 INFO:tasks.ceph.mds.c:Started 2024-05-27T12:35:58.389 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:58.393 INFO:tasks.ceph.mds.a:Started 2024-05-27T12:35:58.393 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:58.396 INFO:tasks.ceph.mds.d:Started 2024-05-27T12:35:58.396 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:58.403 INFO:tasks.ceph.mds.b:Started 2024-05-27T12:35:58.403 DEBUG:teuthology.parallel:result is None 2024-05-27T12:35:58.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-27T12:35:58.539 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-05-27T12:35:58.539 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-05-27T12:35:58.547 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2024-05-27T12:35:58.551 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2024-05-27T12:35:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.581+0000 7fe3c3000640 1 Processor -- start 2024-05-27T12:35:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.582+0000 7fe3c3000640 1 -- start start 2024-05-27T12:35:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc15cb20 0x7fe3bc0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3bc0b1f90 0x7fe3bc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe3bc0ce5a0 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe3bc09b790 con 0x7fe3bc0b1f90 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c3000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc09ba70 con 0x7fe3bc15cb20 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38886/0 (socket says 172.21.15.94:38886) 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 -- 172.21.15.94:0/1200887713 learned_addr learned my addr 172.21.15.94:0/1200887713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3bbe00640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc15cb20 0x7fe3bc0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3b3400640 1 --2- 172.21.15.94:0/1200887713 >> v2:172.21.15.97:3300/0 conn(0x7fe3bc0b1f90 0x7fe3bc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 -- 172.21.15.94:0/1200887713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc15cb20 msgr2=0x7fe3bc0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc15cb20 0x7fe3bc0d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 -- 172.21.15.94:0/1200887713 >> v2:172.21.15.97:3300/0 conn(0x7fe3bc0b1f90 msgr2=0x7fe3bc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.583+0000 7fe3c1000640 1 --2- 172.21.15.94:0/1200887713 >> v2:172.21.15.97:3300/0 conn(0x7fe3bc0b1f90 0x7fe3bc0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.584+0000 7fe3c1000640 1 -- 172.21.15.94:0/1200887713 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc09bd50 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.584+0000 7fe3c1000640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0738b0 tx=0x7fe3b402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fae94a95ec4c0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.585+0000 7fe3bb400640 1 -- 172.21.15.94:0/1200887713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4037070 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.585+0000 7fe3bb400640 1 -- 172.21.15.94:0/1200887713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b402ae00 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.585+0000 7fe3bb400640 1 -- 172.21.15.94:0/1200887713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4032660 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.585+0000 7fe3c3000640 1 -- 172.21.15.94:0/1200887713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 msgr2=0x7fe3bc0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.585+0000 7fe3c3000640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0738b0 tx=0x7fe3b402a0b0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 -- 172.21.15.94:0/1200887713 shutdown_connections 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0ce060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 --2- 172.21.15.94:0/1200887713 >> v2:172.21.15.97:3300/0 conn(0x7fe3bc0b1f90 0x7fe3bc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 --2- 172.21.15.94:0/1200887713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc15cb20 0x7fe3bc0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 -- 172.21.15.94:0/1200887713 >> 172.21.15.94:0/1200887713 conn(0x7fe3bc0b4160 msgr2=0x7fe3bc0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 -- 172.21.15.94:0/1200887713 shutdown_connections 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.586+0000 7fe3c3000640 1 -- 172.21.15.94:0/1200887713 wait complete. 2024-05-27T12:35:58.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.587+0000 7fe3c3000640 1 Processor -- start 2024-05-27T12:35:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.587+0000 7fe3c3000640 1 -- start start 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0b1f90 0x7fe3bc154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc15cb20 0x7fe3bc15f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d0240 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc09b410 con 0x7fe3bc15cb20 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.588+0000 7fe3c3000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d03c0 con 0x7fe3bc0b1f90 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38894/0 (socket says 172.21.15.94:38894) 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 -- 172.21.15.94:0/3796299399 learned_addr learned my addr 172.21.15.94:0/3796299399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3bbe00640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0b1f90 0x7fe3bc154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3c1000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc15cb20 0x7fe3bc15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 -- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0b1f90 msgr2=0x7fe3bc154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0b1f90 0x7fe3bc154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 -- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc15cb20 msgr2=0x7fe3bc15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc15cb20 0x7fe3bc15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 -- 172.21.15.94:0/3796299399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc15e5c0 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.589+0000 7fe3b3400640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400e890 tx=0x7fe3a400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.590+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4017070 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.590+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a400c940 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.590+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400cd10 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.590+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc15e8a0 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.590+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc15ee60 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.591+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc068a70 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.592+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3a401a070 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.593+0000 7fe3b8c00640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:58.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.593+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7fe3a408bbe0 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.594+0000 7fe3bbe00640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:58.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.594+0000 7fe3bbe00640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac011340 tx=0x7fe3ac00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.598+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3a4059b90 con 0x7fe3bc0b23d0 2024-05-27T12:35:58.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:35:58.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:35:58.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:58.810+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe3bc0b4e80 con 0x7fe3bc0b23d0 2024-05-27T12:35:59.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.326+0000 7fe3b8c00640 1 -- 172.21.15.94:0/3796299399 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7fe3a405de90 con 0x7fe3bc0b23d0 2024-05-27T12:35:59.326 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-05-27T12:35:59.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 msgr2=0x7fe39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac011340 tx=0x7fe3ac00b040 comp rx=0 tx=0).stop 2024-05-27T12:35:59.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 msgr2=0x7fe3bc0b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400e890 tx=0x7fe3a400ed50 comp rx=0 tx=0).stop 2024-05-27T12:35:59.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 shutdown_connections 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.330+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe39806c8f0 0x7fe39806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc15cb20 0x7fe3bc15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3bc0b23d0 0x7fe3bc0b7e20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 --2- 172.21.15.94:0/3796299399 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe3bc0b1f90 0x7fe3bc154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 >> 172.21.15.94:0/3796299399 conn(0x7fe3bc0b4160 msgr2=0x7fe3bc070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 shutdown_connections 2024-05-27T12:35:59.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.331+0000 7fe3c3000640 1 -- 172.21.15.94:0/3796299399 wait complete. 2024-05-27T12:35:59.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-27T12:35:59.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.594+0000 7f3d5f200640 1 Processor -- start 2024-05-27T12:35:59.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.594+0000 7f3d5f200640 1 -- start start 2024-05-27T12:35:59.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.595+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d60160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.595+0000 7f3d5f200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3d601460d0 0x7f3d600ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5f200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600ceae0 con 0x7f3d600ce610 2024-05-27T12:35:59.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5f200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3d601602c0 con 0x7f3d601460d0 2024-05-27T12:35:59.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3d601605a0 con 0x7f3d60145cf0 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5e800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3d600ce610 0x7f3d60071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:54102/0 (socket says 172.21.15.94:54102) 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5e800640 1 -- 172.21.15.94:0/3110883913 learned_addr learned my addr 172.21.15.94:0/3110883913 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5d400640 1 --2- 172.21.15.94:0/3110883913 >> v2:172.21.15.94:3300/0 conn(0x7f3d601460d0 0x7f3d600ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5de00640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d60160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5d400640 1 -- 172.21.15.94:0/3110883913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 msgr2=0x7f3d60160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5d400640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d60160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.596+0000 7f3d5d400640 1 -- 172.21.15.94:0/3110883913 >> v1:172.21.15.94:6789/0 conn(0x7f3d600ce610 legacy=0x7f3d60071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.597+0000 7f3d5d400640 1 -- 172.21.15.94:0/3110883913 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d60160880 con 0x7f3d601460d0 2024-05-27T12:35:59.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.597+0000 7f3d5ca00640 1 -- 172.21.15.94:0/3110883913 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4048342425 0 0) 0x7f3d600ceae0 con 0x7f3d600ce610 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5d400640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d600ce0d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0097a0 tx=0x7f3d4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c15bc47fda205be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5ca00640 1 -- 172.21.15.94:0/3110883913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c037070 con 0x7f3d601460d0 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5ca00640 1 -- 172.21.15.94:0/3110883913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c032440 con 0x7f3d601460d0 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5ca00640 1 -- 172.21.15.94:0/3110883913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c032810 con 0x7f3d601460d0 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 -- 172.21.15.94:0/3110883913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 msgr2=0x7f3d600ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d600ce0d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0097a0 tx=0x7f3d4c02a180 comp rx=0 tx=0).stop 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 -- 172.21.15.94:0/3110883913 shutdown_connections 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d600ce0d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 --2- 172.21.15.94:0/3110883913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d60160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 -- 172.21.15.94:0/3110883913 >> 172.21.15.94:0/3110883913 conn(0x7f3d600b2160 msgr2=0x7f3d600b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:59.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 -- 172.21.15.94:0/3110883913 shutdown_connections 2024-05-27T12:35:59.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.598+0000 7f3d5f200640 1 -- 172.21.15.94:0/3110883913 wait complete. 2024-05-27T12:35:59.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.600+0000 7f3d5f200640 1 Processor -- start 2024-05-27T12:35:59.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.600+0000 7f3d5f200640 1 -- start start 2024-05-27T12:35:59.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d600ce610 0x7f3d6015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d601f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6006ec80 con 0x7f3d601460d0 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6006eb00 con 0x7f3d600ce610 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5f200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3d60072470 con 0x7f3d60145cf0 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38914/0 (socket says 172.21.15.94:38914) 2024-05-27T12:35:59.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.601+0000 7f3d5e800640 1 -- 172.21.15.94:0/2960369288 learned_addr learned my addr 172.21.15.94:0/2960369288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:35:59.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 -- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 msgr2=0x7f3d601f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d601f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 -- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d600ce610 msgr2=0x7f3d6015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5de00640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d600ce610 0x7f3d6015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d600ce610 0x7f3d6015cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 -- 172.21.15.94:0/2960369288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6015d280 con 0x7f3d601460d0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5e800640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c780 tx=0x7f3d5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58018070 con 0x7f3d601460d0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5800a640 con 0x7f3d601460d0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800aab0 con 0x7f3d601460d0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.602+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601f3570 con 0x7f3d601460d0 2024-05-27T12:35:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.603+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601f3b30 con 0x7f3d601460d0 2024-05-27T12:35:59.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.604+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d60075460 con 0x7f3d601460d0 2024-05-27T12:35:59.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.605+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5801b070 con 0x7f3d601460d0 2024-05-27T12:35:59.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.606+0000 7f3d56000640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:35:59.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.606+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3d5808ae80 con 0x7f3d601460d0 2024-05-27T12:35:59.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.606+0000 7f3d5de00640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:35:59.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.607+0000 7f3d5de00640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 0x7f3d4006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:35:59.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.610+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d58059560 con 0x7f3d601460d0 2024-05-27T12:35:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.881+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3d601f30c0 con 0x7f3d601460d0 2024-05-27T12:35:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.882+0000 7f3d56000640 1 -- 172.21.15.94:0/2960369288 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f3d58058e60 con 0x7f3d601460d0 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC6fVRmBSGxCRAAlqoZ3k3248qaaawrgczaKA==", 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.882 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC6fVRm4SUSJhAAx+eNO2M0sRsYdhFIUMnnsA==", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-05-27T12:35:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC6fVRmk7H7FxAA5tzg/wo5AlIjKmlnw5Ntcw==", 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC6fVRmrrRsNRAAy5Q0sgtxD/1HDx82gqG/sA==", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-05-27T12:35:59.885 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC9fVRmhwr0EBAAiVA4TQAPBG2hcQ5rvnDwrQ==", 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.886 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDAfVRmkEwZGxAA2GvYAQJ2jCQnyb734JBYsg==", 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDDfVRmC2TlNRAATX3pQr5v3nozFEht3W3ktw==", 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.888 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.889 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDHfVRm/qHnAxAAFRu8l+2+uB+N1rH3D35Rpw==", 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-05-27T12:35:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDNfVRmiKNPExAAm9SFMTwayddMameEe0itjg==", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDQfVRmbBYtHhAASndxPHopciB2VKHisOwsuw==", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.891 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDTfVRmD3PsNxAAaBZs2zwb3XH9djFk+J/ahg==", 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDXfVRmjhL4FRAAhz9w3S1rJYBu9YZCoQpH6A==", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC7fVRm82KsBhAApDuhvkqez8bDlFwJfJwTsw==", 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-05-27T12:35:59.894 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC7fVRma9UOGhAAH6lIxB/+y1s/OV0Rq/UxBw==", 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC7fVRmF/O5EBAA9/DVyfKTzzmOwAtFRf/1og==", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC7fVRm6/nHIxAAUkcRRY+5F3coqLK9hPiX/A==", 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.896 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC5fVRmXhd4BBAAG4utjqJr8ZAUMuKdmn22RA==", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mds", 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRmnMqMJBAAJGVbpqF1hCNoeECPIEv6Tg==", 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mds" 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.898 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mgr", 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRmetSMJBAANkJ1FNzL+N0U+S7Ka/EwXQ==", 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-osd", 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRm69yMJBAAiToi1q5962G60ZaMQFG2XA==", 2024-05-27T12:35:59.899 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-osd" 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd", 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRmheWMJBAAZOy8XaeOxvvIpZulDLIO3g==", 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.900 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRmu+6MJBAA+vh4ZBU9BghPqdIabcxVFQ==", 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rgw", 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDffVRmqveMJBAAevwsKsudFPb7i2cCMX4boQ==", 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.901 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC6fVRmT8gKABAAWwijw+gHiXip0tT2sfaRBg==", 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQC5fVRm9xPrLxAAlqZUU5Xv2T5xYmQ/qfm8VA==", 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-05-27T12:35:59.903 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:35:59.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.884+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 msgr2=0x7f3d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.884+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 0x7f3d4006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.884+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 msgr2=0x7f3d6015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.884+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c780 tx=0x7f3d5800cc40 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 shutdown_connections 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d601460d0 0x7f3d6015ed70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d60145cf0 0x7f3d601f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 --2- 172.21.15.94:0/2960369288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d600ce610 0x7f3d6015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 >> 172.21.15.94:0/2960369288 conn(0x7f3d600b2160 msgr2=0x7f3d60144350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:35:59.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 shutdown_connections 2024-05-27T12:35:59.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:35:59.885+0000 7f3d5f200640 1 -- 172.21.15.94:0/2960369288 wait complete. 2024-05-27T12:35:59.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-27T12:36:00.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.107+0000 7fec63200640 1 Processor -- start 2024-05-27T12:36:00.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.108+0000 7fec63200640 1 -- start start 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.109+0000 7fec63200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec64145df0 0x7fec640cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.109+0000 7fec63200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec64160850 0x7fec640721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.109+0000 7fec63200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec640cdf50 con 0x7fec64071d40 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.109+0000 7fec63200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fec640726f0 con 0x7fec64145df0 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.109+0000 7fec63200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fec64161000 con 0x7fec64160850 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec64145df0 0x7fec640cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fec64145df0 0x7fec640cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47870/0 (socket says 172.21.15.94:47870) 2024-05-27T12:36:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 -- 172.21.15.94:0/1944261271 learned_addr learned my addr 172.21.15.94:0/1944261271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 -- 172.21.15.94:0/1944261271 >> v2:172.21.15.97:3300/0 conn(0x7fec64160850 msgr2=0x7fec640721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 --2- 172.21.15.94:0/1944261271 >> v2:172.21.15.97:3300/0 conn(0x7fec64160850 0x7fec640721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 -- 172.21.15.94:0/1944261271 >> v1:172.21.15.97:6789/0 conn(0x7fec64071d40 legacy=0x7fec64160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.110+0000 7fec61e00640 1 -- 172.21.15.94:0/1944261271 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec64160d20 con 0x7fec64145df0 2024-05-27T12:36:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.111+0000 7fec61e00640 1 --2- 172.21.15.94:0/1944261271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64145df0 0x7fec640cda10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec640d0290 tx=0x7fec5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a64e8b50462bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.111+0000 7fec60a00640 1 -- 172.21.15.94:0/1944261271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54037070 con 0x7fec64145df0 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.111+0000 7fec60a00640 1 -- 172.21.15.94:0/1944261271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec54032440 con 0x7fec64145df0 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.111+0000 7fec60a00640 1 -- 172.21.15.94:0/1944261271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54032810 con 0x7fec64145df0 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 -- 172.21.15.94:0/1944261271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64145df0 msgr2=0x7fec640cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 --2- 172.21.15.94:0/1944261271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64145df0 0x7fec640cda10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec640d0290 tx=0x7fec5402a1a0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 -- 172.21.15.94:0/1944261271 shutdown_connections 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 --2- 172.21.15.94:0/1944261271 >> v2:172.21.15.97:3300/0 conn(0x7fec64160850 0x7fec640721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 --2- 172.21.15.94:0/1944261271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64145df0 0x7fec640cda10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 -- 172.21.15.94:0/1944261271 >> 172.21.15.94:0/1944261271 conn(0x7fec6406b930 msgr2=0x7fec6406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 -- 172.21.15.94:0/1944261271 shutdown_connections 2024-05-27T12:36:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.112+0000 7fec63200640 1 -- 172.21.15.94:0/1944261271 wait complete. 2024-05-27T12:36:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.114+0000 7fec63200640 1 Processor -- start 2024-05-27T12:36:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.114+0000 7fec63200640 1 -- start start 2024-05-27T12:36:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec64071d40 0x7fec64132f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec64145df0 0x7fec64131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec640cd4f0 con 0x7fec64160850 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fec640cd210 con 0x7fec64071d40 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec63200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fec641461d0 con 0x7fec64145df0 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.115+0000 7fec62800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47884/0 (socket says 172.21.15.94:47884) 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 -- 172.21.15.94:0/1202249766 learned_addr learned my addr 172.21.15.94:0/1202249766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec61e00640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec64071d40 0x7fec64132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec61400640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec64145df0 0x7fec64131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 -- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec64145df0 msgr2=0x7fec64131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec64145df0 0x7fec64131680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 -- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec64071d40 msgr2=0x7fec64132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec64071d40 0x7fec64132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 -- 172.21.15.94:0/1202249766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641334b0 con 0x7fec64160850 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.116+0000 7fec62800640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00b9e0 tx=0x7fec5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.117+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c019070 con 0x7fec64160850 2024-05-27T12:36:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.117+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5c00cb20 con 0x7fec64160850 2024-05-27T12:36:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.117+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec64131ed0 con 0x7fec64160850 2024-05-27T12:36:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.117+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c014440 con 0x7fec64160850 2024-05-27T12:36:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.117+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec64132770 con 0x7fec64160850 2024-05-27T12:36:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.118+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec64075460 con 0x7fec64160850 2024-05-27T12:36:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.119+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec5c01c070 con 0x7fec64160850 2024-05-27T12:36:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.120+0000 7fec5a000640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 0x7fec4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.120+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fec5c08b580 con 0x7fec64160850 2024-05-27T12:36:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.121+0000 7fec61e00640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 0x7fec4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.122+0000 7fec61e00640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 0x7fec4406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec54034000 tx=0x7fec5402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.125+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec5c059c60 con 0x7fec64160850 2024-05-27T12:36:00.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.399+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fec6415e520 con 0x7fec64160850 2024-05-27T12:36:00.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.404+0000 7fec5a000640 1 -- 172.21.15.94:0/1202249766 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fec5c059560 con 0x7fec64160850 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 msgr2=0x7fec4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 0x7fec4406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec54034000 tx=0x7fec5402e000 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 msgr2=0x7fec640b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00b9e0 tx=0x7fec5c00bea0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 shutdown_connections 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fec4406c920 0x7fec4406ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec64160850 0x7fec640b34c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec64145df0 0x7fec64131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 --2- 172.21.15.94:0/1202249766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec64071d40 0x7fec64132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 >> 172.21.15.94:0/1202249766 conn(0x7fec6406b930 msgr2=0x7fec640b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:00.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 shutdown_connections 2024-05-27T12:36:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.406+0000 7fec63200640 1 -- 172.21.15.94:0/1202249766 wait complete. 2024-05-27T12:36:00.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-27T12:36:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.611+0000 7fcaeae00640 1 Processor -- start 2024-05-27T12:36:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.612+0000 7fcaeae00640 1 -- start start 2024-05-27T12:36:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaeae00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae4162170 0x7fcae40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaeae00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fcae40d15a0 0x7fcae40d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaeae00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae40d2a80 con 0x7fcae40b34e0 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaeae00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fcae40d3650 con 0x7fcae40d15a0 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaeae00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcae40d2d20 con 0x7fcae4162170 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcae9000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fcae40b34e0 0x7fcae40b41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32976/0 (socket says 172.21.15.94:32976) 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcae9000640 1 -- 172.21.15.94:0/3650960518 learned_addr learned my addr 172.21.15.94:0/3650960518 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.613+0000 7fcaea400640 1 --2- 172.21.15.94:0/3650960518 >> v2:172.21.15.94:3300/0 conn(0x7fcae40d15a0 0x7fcae40d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcae9a00640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae4162170 0x7fcae40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2953723522 0 0) 0x7fcae40d2a80 con 0x7fcae40b34e0 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcacc003520 con 0x7fcae40b34e0 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcaea400640 1 -- 172.21.15.94:0/3650960518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae4162170 msgr2=0x7fcae40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcaea400640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae4162170 0x7fcae40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcaea400640 1 -- 172.21.15.94:0/3650960518 >> v1:172.21.15.94:6789/0 conn(0x7fcae40b34e0 legacy=0x7fcae40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1274140546 0 0) 0x7fcacc003520 con 0x7fcae40b34e0 2024-05-27T12:36:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.614+0000 7fcaea400640 1 -- 172.21.15.94:0/3650960518 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40d3000 con 0x7fcae40d15a0 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.615+0000 7fcaea400640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40d15a0 0x7fcae40d02b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538fac42acaf2f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.615+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc037070 con 0x7fcae40d15a0 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.615+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc02aba0 con 0x7fcae40d15a0 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.615+0000 7fcae3e00640 1 -- 172.21.15.94:0/3650960518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad80019f0 con 0x7fcae40d15a0 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 -- 172.21.15.94:0/3650960518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40d15a0 msgr2=0x7fcae40d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40d15a0 0x7fcae40d02b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02a110 comp rx=0 tx=0).stop 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 -- 172.21.15.94:0/3650960518 shutdown_connections 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40d15a0 0x7fcae40d02b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 --2- 172.21.15.94:0/3650960518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae4162170 0x7fcae40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 -- 172.21.15.94:0/3650960518 >> 172.21.15.94:0/3650960518 conn(0x7fcae408bfc0 msgr2=0x7fcae40d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 -- 172.21.15.94:0/3650960518 shutdown_connections 2024-05-27T12:36:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.616+0000 7fcaeae00640 1 -- 172.21.15.94:0/3650960518 wait complete. 2024-05-27T12:36:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.618+0000 7fcaeae00640 1 Processor -- start 2024-05-27T12:36:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.618+0000 7fcaeae00640 1 -- start start 2024-05-27T12:36:00.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae40d15a0 0x7fcae414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae4162170 0x7fcae40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40b48d0 con 0x7fcae40b34e0 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d07f0 con 0x7fcae4162170 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcaeae00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fcae40b4a50 con 0x7fcae40d15a0 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47898/0 (socket says 172.21.15.94:47898) 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 -- 172.21.15.94:0/457579082 learned_addr learned my addr 172.21.15.94:0/457579082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 -- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae40d15a0 msgr2=0x7fcae414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae40d15a0 0x7fcae414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 -- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae4162170 msgr2=0x7fcae40c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae4162170 0x7fcae40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.619+0000 7fcae9a00640 1 -- 172.21.15.94:0/457579082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae414e000 con 0x7fcae40b34e0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcae9a00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d5f0 tx=0x7fcad400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40125f0 con 0x7fcae40b34e0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4019070 con 0x7fcae40b34e0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae415c850 con 0x7fcae40b34e0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40129d0 con 0x7fcae40b34e0 2024-05-27T12:36:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.620+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae415ce10 con 0x7fcae40b34e0 2024-05-27T12:36:00.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.621+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae4075460 con 0x7fcae40b34e0 2024-05-27T12:36:00.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.622+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcad4012b70 con 0x7fcae40b34e0 2024-05-27T12:36:00.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.623+0000 7fcae1600640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.623+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcad408bc20 con 0x7fcae40b34e0 2024-05-27T12:36:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.624+0000 7fcae9000640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.625+0000 7fcae9000640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 0x7fcac806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcad80012e0 tx=0x7fcad80013e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.628+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcad405a390 con 0x7fcae40b34e0 2024-05-27T12:36:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.901+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fcae40b42f0 con 0x7fcae40b34e0 2024-05-27T12:36:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.906+0000 7fcae1600640 1 -- 172.21.15.94:0/457579082 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fcad4059c90 con 0x7fcae40b34e0 2024-05-27T12:36:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 msgr2=0x7fcac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 0x7fcac806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcad80012e0 tx=0x7fcad80013e0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 msgr2=0x7fcae415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcad400d5f0 tx=0x7fcad400dab0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 shutdown_connections 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fcac806c8f0 0x7fcac806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae4162170 0x7fcae40c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcae40d15a0 0x7fcae414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 --2- 172.21.15.94:0/457579082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40b34e0 0x7fcae415d8f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 >> 172.21.15.94:0/457579082 conn(0x7fcae408bfc0 msgr2=0x7fcae40b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 shutdown_connections 2024-05-27T12:36:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:00.909+0000 7fcaeae00640 1 -- 172.21.15.94:0/457579082 wait complete. 2024-05-27T12:36:00.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-27T12:36:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.127+0000 7f7d64800640 1 Processor -- start 2024-05-27T12:36:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.127+0000 7f7d64800640 1 -- start start 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d64800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d5c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d64800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d5c0d1d70 0x7f7d5c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d64800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5c15eee0 con 0x7f7d5c15eb00 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d64800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7d5c0d5700 con 0x7f7d5c0d1d70 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d64800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d21a0 con 0x7f7d5c0cfb40 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d61e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7d5c15eb00 0x7f7d5c0d3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32980/0 (socket says 172.21.15.94:32980) 2024-05-27T12:36:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.129+0000 7f7d61e00640 1 -- 172.21.15.94:0/4279639087 learned_addr learned my addr 172.21.15.94:0/4279639087 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d62800640 1 --2- 172.21.15.94:0/4279639087 >> v2:172.21.15.97:3300/0 conn(0x7f7d5c0d1d70 0x7f7d5c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d61400640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d5c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021002358 0 0) 0x7f7d5c15eee0 con 0x7f7d5c15eb00 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d48003520 con 0x7f7d5c15eb00 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2008441738 0 0) 0x7f7d48003520 con 0x7f7d5c15eb00 2024-05-27T12:36:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 msgr2=0x7f7d5c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d5c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 >> v2:172.21.15.97:3300/0 conn(0x7f7d5c0d1d70 msgr2=0x7f7d5c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 --2- 172.21.15.94:0/4279639087 >> v2:172.21.15.97:3300/0 conn(0x7f7d5c0d1d70 0x7f7d5c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0d4660 con 0x7f7d5c15eb00 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f7d540019f0 con 0x7f7d5c15eb00 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.130+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 >> v1:172.21.15.94:6789/0 conn(0x7f7d5c15eb00 legacy=0x7f7d5c0d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d48003f60 0x7f7d48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d48004d40 0x7f7d480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d5700 con 0x7f7d48003f60 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d21a0 con 0x7f7d48004d40 2024-05-27T12:36:01.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4800ba40 con 0x7f7d5c0cfb40 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d61400640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d48003f60 0x7f7d48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.131+0000 7f7d61e00640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d62800640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d48004d40 0x7f7d480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d61400640 1 -- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 msgr2=0x7f7d480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d61400640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5c0cfb40 0x7f7d480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d61400640 1 -- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d48004d40 msgr2=0x7f7d480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d61400640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d48004d40 0x7f7d480051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.132+0000 7f7d61400640 1 -- 172.21.15.94:0/4279639087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48003520 con 0x7f7d48003f60 2024-05-27T12:36:01.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.133+0000 7f7d61400640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d48003f60 0x7f7d48004800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009ac0 tx=0x7f7d48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a786ea6e7702023 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.133+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c036070 con 0x7f7d48003f60 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.133+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d4c031400 con 0x7f7d48003f60 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.133+0000 7f7d60a00640 1 -- 172.21.15.94:0/4279639087 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c009690 con 0x7f7d48003f60 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 -- 172.21.15.94:0/4279639087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d48003f60 msgr2=0x7f7d48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 --2- 172.21.15.94:0/4279639087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d48003f60 0x7f7d48004800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009ac0 tx=0x7f7d48005830 comp rx=0 tx=0).stop 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d62800640 1 -- 172.21.15.94:0/4279639087 reap_dead start 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 -- 172.21.15.94:0/4279639087 shutdown_connections 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 -- 172.21.15.94:0/4279639087 >> 172.21.15.94:0/4279639087 conn(0x7f7d5c067a50 msgr2=0x7f7d5c067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 -- 172.21.15.94:0/4279639087 shutdown_connections 2024-05-27T12:36:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.134+0000 7f7d64800640 1 -- 172.21.15.94:0/4279639087 wait complete. 2024-05-27T12:36:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.136+0000 7f7d64800640 1 Processor -- start 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.136+0000 7f7d64800640 1 -- start start 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d48003f60 0x7f7d5c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c0b9280 0x7f7d5c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d4f00 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d4c20 con 0x7f7d5c0b9280 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d64800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d51e0 con 0x7f7d48003f60 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d61400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d61400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47910/0 (socket says 172.21.15.94:47910) 2024-05-27T12:36:01.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.137+0000 7f7d61400640 1 -- 172.21.15.94:0/1356597510 learned_addr learned my addr 172.21.15.94:0/1356597510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:01.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d62800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c0b9280 0x7f7d5c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61e00640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d48003f60 0x7f7d5c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 -- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d48003f60 msgr2=0x7f7d5c0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d48003f60 0x7f7d5c0c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 -- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c0b9280 msgr2=0x7f7d5c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c0b9280 0x7f7d5c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 -- 172.21.15.94:0/1356597510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0ca0a0 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d61400640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c00 tx=0x7f7d4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c040020 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d4c003c60 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c036040 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.138+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c0ca380 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.139+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c0ca940 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.140+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c0b9690 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.141+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d4c043020 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.141+0000 7f7d52000640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.142+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f7d4c0abbd0 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.142+0000 7f7d61e00640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.143+0000 7f7d61e00640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 0x7f7d4006ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7d540013d0 tx=0x7f7d540014f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.146+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d5c0b9690 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.397+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f7d5c06bcc0 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.401+0000 7f7d52000640 1 -- 172.21.15.94:0/1356597510 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f7d4c04a030 con 0x7f7d5c0c9bd0 2024-05-27T12:36:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.403+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 msgr2=0x7f7d4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.403+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 0x7f7d4006ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7d540013d0 tx=0x7f7d540014f0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.403+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 msgr2=0x7f7d5c0b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.403+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c00 tx=0x7f7d4c004730 comp rx=0 tx=0).stop 2024-05-27T12:36:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 shutdown_connections 2024-05-27T12:36:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d5c0b9280 0x7f7d5c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d5c0c9bd0 0x7f7d5c0b8d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 --2- 172.21.15.94:0/1356597510 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d48003f60 0x7f7d5c0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 >> 172.21.15.94:0/1356597510 conn(0x7f7d5c067a50 msgr2=0x7f7d5c067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:01.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 shutdown_connections 2024-05-27T12:36:01.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.404+0000 7f7d64800640 1 -- 172.21.15.94:0/1356597510 wait complete. 2024-05-27T12:36:01.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-27T12:36:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.613+0000 7f7712c00640 1 Processor -- start 2024-05-27T12:36:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.614+0000 7f7712c00640 1 -- start start 2024-05-27T12:36:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7712c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f770c0b51d0 0x7f770c169770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7712c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770c0d2a80 con 0x7f770c162170 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7712c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770c0d2d20 con 0x7f770c0d1770 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7712c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f770c0d3650 con 0x7f770c0b51d0 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7711800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f770c162170 0x7f770c0d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32982/0 (socket says 172.21.15.94:32982) 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7711800640 1 -- 172.21.15.94:0/2878159531 learned_addr learned my addr 172.21.15.94:0/2878159531 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.615+0000 7f7712200640 1 --2- 172.21.15.94:0/2878159531 >> v2:172.21.15.94:3300/0 conn(0x7f770c0b51d0 0x7f770c169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.616+0000 7f7712200640 1 -- 172.21.15.94:0/2878159531 >> v1:172.21.15.97:6789/0 conn(0x7f770c0d1770 legacy=0x7f770c0cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.616+0000 7f7712200640 1 -- 172.21.15.94:0/2878159531 >> v1:172.21.15.94:6789/0 conn(0x7f770c162170 legacy=0x7f770c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.616+0000 7f7712200640 1 -- 172.21.15.94:0/2878159531 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0d3000 con 0x7f770c0b51d0 2024-05-27T12:36:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.617+0000 7f7712200640 1 --2- 172.21.15.94:0/2878159531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c169770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77040097a0 tx=0x7f770402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672364b56cc5af01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.617+0000 7f770be00640 1 -- 172.21.15.94:0/2878159531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704037070 con 0x7f770c0b51d0 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.617+0000 7f770be00640 1 -- 172.21.15.94:0/2878159531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770402ae50 con 0x7f770c0b51d0 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.617+0000 7f770be00640 1 -- 172.21.15.94:0/2878159531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704032660 con 0x7f770c0b51d0 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 -- 172.21.15.94:0/2878159531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 msgr2=0x7f770c169770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 --2- 172.21.15.94:0/2878159531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c169770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77040097a0 tx=0x7f770402a140 comp rx=0 tx=0).stop 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 -- 172.21.15.94:0/2878159531 shutdown_connections 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 --2- 172.21.15.94:0/2878159531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c169770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 -- 172.21.15.94:0/2878159531 >> 172.21.15.94:0/2878159531 conn(0x7f770c08bfc0 msgr2=0x7f770c0d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 -- 172.21.15.94:0/2878159531 shutdown_connections 2024-05-27T12:36:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.618+0000 7f7712c00640 1 -- 172.21.15.94:0/2878159531 wait complete. 2024-05-27T12:36:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.620+0000 7f7712c00640 1 Processor -- start 2024-05-27T12:36:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.620+0000 7f7712c00640 1 -- start start 2024-05-27T12:36:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f770c0d1770 0x7f770c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770c162170 0x7f770c0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0b3660 con 0x7f770c0b51d0 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0b34e0 con 0x7f770c162170 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7712c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f770c0b37e0 con 0x7f770c0d1770 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47922/0 (socket says 172.21.15.94:47922) 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 -- 172.21.15.94:0/989396124 learned_addr learned my addr 172.21.15.94:0/989396124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 -- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f770c0d1770 msgr2=0x7f770c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f770c0d1770 0x7f770c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 -- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770c162170 msgr2=0x7f770c0b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770c162170 0x7f770c0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.621+0000 7f7711800640 1 -- 172.21.15.94:0/989396124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c15b650 con 0x7f770c0b51d0 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7711800640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c3c0 tx=0x7f76fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc017070 con 0x7f770c0b51d0 2024-05-27T12:36:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc005ce0 con 0x7f770c0b51d0 2024-05-27T12:36:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc007310 con 0x7f770c0b51d0 2024-05-27T12:36:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c0c92e0 con 0x7f770c0b51d0 2024-05-27T12:36:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.622+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c0c98a0 con 0x7f770c0b51d0 2024-05-27T12:36:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.623+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c075460 con 0x7f770c0b51d0 2024-05-27T12:36:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.624+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76fc01a070 con 0x7f770c0b51d0 2024-05-27T12:36:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.625+0000 7f7709600640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 0x7f76ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.625+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f76fc08ab70 con 0x7f770c0b51d0 2024-05-27T12:36:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.626+0000 7f7710e00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 0x7f76ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:01.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.627+0000 7f7710e00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 0x7f76ec06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f770c0b2ed0 tx=0x7f76f8007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:01.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.630+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76fc0592e0 con 0x7f770c0b51d0 2024-05-27T12:36:01.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.902+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f770c0d0f10 con 0x7f770c0b51d0 2024-05-27T12:36:01.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.906+0000 7f7709600640 1 -- 172.21.15.94:0/989396124 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f76fc058be0 con 0x7f770c0b51d0 2024-05-27T12:36:01.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 msgr2=0x7f76ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 0x7f76ec06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f770c0b2ed0 tx=0x7f76f8007d30 comp rx=0 tx=0).stop 2024-05-27T12:36:01.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 msgr2=0x7f770c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:01.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c3c0 tx=0x7f76fc00c880 comp rx=0 tx=0).stop 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 shutdown_connections 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f76ec06c8f0 0x7f76ec06edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770c162170 0x7f770c0b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f770c0d1770 0x7f770c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 --2- 172.21.15.94:0/989396124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f770c0b51d0 0x7f770c0b1f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.908+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 >> 172.21.15.94:0/989396124 conn(0x7f770c08bfc0 msgr2=0x7f770c160770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:01.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.909+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 shutdown_connections 2024-05-27T12:36:01.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:01.909+0000 7f7712c00640 1 -- 172.21.15.94:0/989396124 wait complete. 2024-05-27T12:36:01.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-27T12:36:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.124+0000 7f86ea800640 1 Processor -- start 2024-05-27T12:36:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.125+0000 7f86ea800640 1 -- start start 2024-05-27T12:36:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.126+0000 7f86ea800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e40b3f70 0x7f86e406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.126+0000 7f86ea800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 0x7f86e4072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.126+0000 7f86ea800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e415cfd0 con 0x7f86e4146060 2024-05-27T12:36:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.126+0000 7f86ea800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f86e415dbe0 con 0x7f86e40b3f70 2024-05-27T12:36:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.126+0000 7f86ea800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f86e406c2c0 con 0x7f86e406be70 2024-05-27T12:36:02.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.127+0000 7f86dbe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e40b3f70 0x7f86e406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.127+0000 7f86e3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f86e4146060 0x7f86e40cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.94:41472/0 (socket says 172.21.15.94:41472) 2024-05-27T12:36:02.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.127+0000 7f86e3400640 1 -- 172.21.15.94:0/3222242969 learned_addr learned my addr 172.21.15.94:0/3222242969 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.127+0000 7f86e3e00640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 0x7f86e4072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86e2a00640 1 -- 172.21.15.94:0/3222242969 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2153014296 0 0) 0x7f86e415cfd0 con 0x7f86e4146060 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86e2a00640 1 -- 172.21.15.94:0/3222242969 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86c8003520 con 0x7f86e4146060 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86dbe00640 1 -- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 msgr2=0x7f86e4072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86dbe00640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 0x7f86e4072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86dbe00640 1 -- 172.21.15.94:0/3222242969 >> v1:172.21.15.97:6789/0 conn(0x7f86e4146060 legacy=0x7f86e40cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86dbe00640 1 -- 172.21.15.94:0/3222242969 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e415d2b0 con 0x7f86e40b3f70 2024-05-27T12:36:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.128+0000 7f86e3e00640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 0x7f86e4072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:36:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.129+0000 7f86dbe00640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40b3f70 0x7f86e406b930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097a0 tx=0x7f86cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc41eea8c854b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.130+0000 7f86e2a00640 1 -- 172.21.15.94:0/3222242969 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc037070 con 0x7f86e40b3f70 2024-05-27T12:36:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.130+0000 7f86e2a00640 1 -- 172.21.15.94:0/3222242969 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc032440 con 0x7f86e40b3f70 2024-05-27T12:36:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.130+0000 7f86e2a00640 1 -- 172.21.15.94:0/3222242969 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc032810 con 0x7f86e40b3f70 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 -- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40b3f70 msgr2=0x7f86e406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40b3f70 0x7f86e406b930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097a0 tx=0x7f86cc02a180 comp rx=0 tx=0).stop 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 -- 172.21.15.94:0/3222242969 shutdown_connections 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e406be70 0x7f86e4072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 --2- 172.21.15.94:0/3222242969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40b3f70 0x7f86e406b930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 -- 172.21.15.94:0/3222242969 >> 172.21.15.94:0/3222242969 conn(0x7f86e40b2240 msgr2=0x7f86e40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 -- 172.21.15.94:0/3222242969 shutdown_connections 2024-05-27T12:36:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.131+0000 7f86ea800640 1 -- 172.21.15.94:0/3222242969 wait complete. 2024-05-27T12:36:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.133+0000 7f86ea800640 1 Processor -- start 2024-05-27T12:36:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.133+0000 7f86ea800640 1 -- start start 2024-05-27T12:36:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 0x7f86e413b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40b3f70 0x7f86e413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86e40cdfb0 con 0x7f86e4146060 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86e415d870 con 0x7f86e406be70 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86ea800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f86e4146440 con 0x7f86e40b3f70 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47928/0 (socket says 172.21.15.94:47928) 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.134+0000 7f86e3e00640 1 -- 172.21.15.94:0/2720033529 learned_addr learned my addr 172.21.15.94:0/2720033529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86dbe00640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40b3f70 0x7f86e413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3400640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 0x7f86e413b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 -- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40b3f70 msgr2=0x7f86e413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40b3f70 0x7f86e413c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 -- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 msgr2=0x7f86e413b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 0x7f86e413b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 -- 172.21.15.94:0/2720033529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e413ccf0 con 0x7f86e4146060 2024-05-27T12:36:02.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3400640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 0x7f86e413b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86e3e00640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009f00 tx=0x7f86dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc018030 con 0x7f86e4146060 2024-05-27T12:36:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86dc00ce50 con 0x7f86e4146060 2024-05-27T12:36:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.135+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc003bb0 con 0x7f86e4146060 2024-05-27T12:36:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.136+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e413cfd0 con 0x7f86e4146060 2024-05-27T12:36:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.136+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e413d590 con 0x7f86e4146060 2024-05-27T12:36:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.137+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e40d1120 con 0x7f86e4146060 2024-05-27T12:36:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.138+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86dc01b070 con 0x7f86e4146060 2024-05-27T12:36:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.139+0000 7f86db400640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.139+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f86dc08c260 con 0x7f86e4146060 2024-05-27T12:36:02.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.140+0000 7f86e3400640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.141+0000 7f86e3400640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d40011e0 tx=0x7f86d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.144+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86e40d1120 con 0x7f86e4146060 2024-05-27T12:36:02.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.413+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f86e4140370 con 0x7f86e4146060 2024-05-27T12:36:02.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.417+0000 7f86db400640 1 -- 172.21.15.94:0/2720033529 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f86e4140370 con 0x7f86e4146060 2024-05-27T12:36:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.419+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 msgr2=0x7f86c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.419+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 0x7f86c006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d40011e0 tx=0x7f86d4009040 comp rx=0 tx=0).stop 2024-05-27T12:36:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 msgr2=0x7f86e40ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009f00 tx=0x7f86dc00c490 comp rx=0 tx=0).stop 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 shutdown_connections 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f86c006c920 0x7f86c006ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e4146060 0x7f86e40ae8f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40b3f70 0x7f86e413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 --2- 172.21.15.94:0/2720033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e406be70 0x7f86e413b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 >> 172.21.15.94:0/2720033529 conn(0x7f86e40b2240 msgr2=0x7f86e415eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 shutdown_connections 2024-05-27T12:36:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.420+0000 7f86ea800640 1 -- 172.21.15.94:0/2720033529 wait complete. 2024-05-27T12:36:02.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-27T12:36:02.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.635+0000 7fbc7d200640 1 Processor -- start 2024-05-27T12:36:02.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.635+0000 7fbc7d200640 1 -- start start 2024-05-27T12:36:02.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc7d200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbc78146060 0x7fbc780b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc7d200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7806c040 0x7fbc78072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc7d200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7815cfd0 con 0x7fbc780cd8a0 2024-05-27T12:36:02.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc7d200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7815dbe0 con 0x7fbc78146060 2024-05-27T12:36:02.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc7d200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7815d2b0 con 0x7fbc7806c040 2024-05-27T12:36:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc75e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbc78146060 0x7fbc780b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc6fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbc780cd8a0 0x7fbc7806b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32984/0 (socket says 172.21.15.94:32984) 2024-05-27T12:36:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc6fe00640 1 -- 172.21.15.94:0/3008805170 learned_addr learned my addr 172.21.15.94:0/3008805170 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.637+0000 7fbc76800640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7806c040 0x7fbc78072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.638+0000 7fbc75e00640 1 -- 172.21.15.94:0/3008805170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7806c040 msgr2=0x7fbc78072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:02.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.638+0000 7fbc75e00640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7806c040 0x7fbc78072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.638+0000 7fbc75e00640 1 -- 172.21.15.94:0/3008805170 >> v1:172.21.15.94:6789/0 conn(0x7fbc780cd8a0 legacy=0x7fbc7806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.638+0000 7fbc75400640 1 -- 172.21.15.94:0/3008805170 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578131249 0 0) 0x7fbc7815cfd0 con 0x7fbc780cd8a0 2024-05-27T12:36:02.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.638+0000 7fbc75e00640 1 -- 172.21.15.94:0/3008805170 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7815d590 con 0x7fbc78146060 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc75e00640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc78146060 0x7fbc780b3f70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5a5c5ff967a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc75400640 1 -- 172.21.15.94:0/3008805170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037070 con 0x7fbc78146060 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc75400640 1 -- 172.21.15.94:0/3008805170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68032440 con 0x7fbc78146060 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc75400640 1 -- 172.21.15.94:0/3008805170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68032810 con 0x7fbc78146060 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc7d200640 1 -- 172.21.15.94:0/3008805170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc78146060 msgr2=0x7fbc780b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.639+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc78146060 0x7fbc780b3f70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a180 comp rx=0 tx=0).stop 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 -- 172.21.15.94:0/3008805170 shutdown_connections 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7806c040 0x7fbc78072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3008805170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc78146060 0x7fbc780b3f70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 -- 172.21.15.94:0/3008805170 >> 172.21.15.94:0/3008805170 conn(0x7fbc780b2240 msgr2=0x7fbc780b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:02.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 -- 172.21.15.94:0/3008805170 shutdown_connections 2024-05-27T12:36:02.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.640+0000 7fbc7d200640 1 -- 172.21.15.94:0/3008805170 wait complete. 2024-05-27T12:36:02.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.641+0000 7fbc7d200640 1 Processor -- start 2024-05-27T12:36:02.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.642+0000 7fbc7d200640 1 -- start start 2024-05-27T12:36:02.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.642+0000 7fbc7d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.642+0000 7fbc7d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc780cd8a0 0x7fbc78134200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.642+0000 7fbc7d200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc78146060 0x7fbc780ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc7d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780d0110 con 0x7fbc7806c040 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc7d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78146440 con 0x7fbc780cd8a0 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc7d200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fbc780cdd30 con 0x7fbc78146060 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47946/0 (socket says 172.21.15.94:47946) 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 -- 172.21.15.94:0/3362641334 learned_addr learned my addr 172.21.15.94:0/3362641334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 -- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc78146060 msgr2=0x7fbc780ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc78146060 0x7fbc780ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 -- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc780cd8a0 msgr2=0x7fbc78134200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc780cd8a0 0x7fbc78134200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 -- 172.21.15.94:0/3362641334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc78134950 con 0x7fbc7806c040 2024-05-27T12:36:02.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.643+0000 7fbc75e00640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.644+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037040 con 0x7fbc7806c040 2024-05-27T12:36:02.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.644+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68042070 con 0x7fbc7806c040 2024-05-27T12:36:02.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.644+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68030a70 con 0x7fbc7806c040 2024-05-27T12:36:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.644+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc78134c30 con 0x7fbc7806c040 2024-05-27T12:36:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.644+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781351f0 con 0x7fbc7806c040 2024-05-27T12:36:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.645+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc780b5aa0 con 0x7fbc7806c040 2024-05-27T12:36:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.646+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc68030c10 con 0x7fbc7806c040 2024-05-27T12:36:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.647+0000 7fbc6e000640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 0x7fbc5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.647+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbc680ac250 con 0x7fbc7806c040 2024-05-27T12:36:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.647+0000 7fbc6fe00640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 0x7fbc5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.648+0000 7fbc6fe00640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 0x7fbc5406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007fd0 tx=0x7fbc60007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:02.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.651+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc780b5aa0 con 0x7fbc7806c040 2024-05-27T12:36:02.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.917+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fbc78068a70 con 0x7fbc7806c040 2024-05-27T12:36:02.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.922+0000 7fbc6e000640 1 -- 172.21.15.94:0/3362641334 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fbc68045360 con 0x7fbc7806c040 2024-05-27T12:36:02.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 msgr2=0x7fbc5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 0x7fbc5406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007fd0 tx=0x7fbc60007920 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 msgr2=0x7fbc781e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc680046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 shutdown_connections 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbc5406c8f0 0x7fbc5406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc780cd8a0 0x7fbc78134200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc78146060 0x7fbc780ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 --2- 172.21.15.94:0/3362641334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc7806c040 0x7fbc781e8580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:02.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 >> 172.21.15.94:0/3362641334 conn(0x7fbc780b2240 msgr2=0x7fbc780d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:02.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 shutdown_connections 2024-05-27T12:36:02.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:02.924+0000 7fbc7d200640 1 -- 172.21.15.94:0/3362641334 wait complete. 2024-05-27T12:36:02.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:36:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.126+0000 7f23e2000640 1 Processor -- start 2024-05-27T12:36:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.126+0000 7f23e2000640 1 -- start start 2024-05-27T12:36:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23e2000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23dc145cf0 0x7f23dc160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23e2000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23dc0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23e2000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc0ceae0 con 0x7f23dc1460d0 2024-05-27T12:36:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23e2000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc1602c0 con 0x7f23dc145cf0 2024-05-27T12:36:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23e2000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc1605a0 con 0x7f23dc071d60 2024-05-27T12:36:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23dbe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f23dc1460d0 0x7f23dc0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32988/0 (socket says 172.21.15.94:32988) 2024-05-27T12:36:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.127+0000 7f23dbe00640 1 -- 172.21.15.94:0/697697637 learned_addr learned my addr 172.21.15.94:0/697697637 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205567352 0 0) 0x7f23dc0ceae0 con 0x7f23dc1460d0 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23e0c00640 1 --2- 172.21.15.94:0/697697637 >> v2:172.21.15.97:3300/0 conn(0x7f23dc145cf0 0x7f23dc160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23e1600640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23dc0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c0003520 con 0x7f23dc1460d0 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 979149446 0 0) 0x7f23c0003520 con 0x7f23dc1460d0 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 msgr2=0x7f23dc0721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23dc0721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 >> v2:172.21.15.97:3300/0 conn(0x7f23dc145cf0 msgr2=0x7f23dc160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 --2- 172.21.15.94:0/697697637 >> v2:172.21.15.97:3300/0 conn(0x7f23dc145cf0 0x7f23dc160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc160880 con 0x7f23dc1460d0 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f23c8001670 con 0x7f23dc1460d0 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 >> v1:172.21.15.94:6789/0 conn(0x7f23dc1460d0 legacy=0x7f23dc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0004800 0x7f23c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1602c0 con 0x7f23c00039b0 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1605a0 con 0x7f23c0004800 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.128+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f23c000baa0 con 0x7f23dc071d60 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23dbe00640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0004800 0x7f23c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e1600640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 -- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 msgr2=0x7f23c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc071d60 0x7f23c00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 -- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0004800 msgr2=0x7f23c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23c0004800 0x7f23c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.129+0000 7f23e0c00640 1 -- 172.21.15.94:0/697697637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c0003520 con 0x7f23c00039b0 2024-05-27T12:36:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.130+0000 7f23e0c00640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00ba00 tx=0x7f23c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2b51802f3d70ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc030070 con 0x7f23c00039b0 2024-05-27T12:36:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc02adb0 con 0x7f23c00039b0 2024-05-27T12:36:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23db400640 1 -- 172.21.15.94:0/697697637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00b600 con 0x7f23c00039b0 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 -- 172.21.15.94:0/697697637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 msgr2=0x7f23c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 --2- 172.21.15.94:0/697697637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00ba00 tx=0x7f23c0005830 comp rx=0 tx=0).stop 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e1600640 1 -- 172.21.15.94:0/697697637 reap_dead start 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 -- 172.21.15.94:0/697697637 shutdown_connections 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 -- 172.21.15.94:0/697697637 >> 172.21.15.94:0/697697637 conn(0x7f23dc0b2160 msgr2=0x7f23dc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 -- 172.21.15.94:0/697697637 shutdown_connections 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.131+0000 7f23e2000640 1 -- 172.21.15.94:0/697697637 wait complete. 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.133+0000 7f23e2000640 1 Processor -- start 2024-05-27T12:36:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.133+0000 7f23e2000640 1 -- start start 2024-05-27T12:36:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23dc0b4500 0x7f23dc15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc0af130 0x7f23dc0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc06eb00 con 0x7f23c00039b0 2024-05-27T12:36:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0cedc0 con 0x7f23dc0b4500 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e2000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc06ec80 con 0x7f23dc0af130 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47960/0 (socket says 172.21.15.94:47960) 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.134+0000 7f23e0c00640 1 -- 172.21.15.94:0/2623417194 learned_addr learned my addr 172.21.15.94:0/2623417194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e1600640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc0af130 0x7f23dc0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 -- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc0af130 msgr2=0x7f23dc0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc0af130 0x7f23dc0af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 -- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23dc0b4500 msgr2=0x7f23dc15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23dc0b4500 0x7f23dc15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 -- 172.21.15.94:0/2623417194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc13fcd0 con 0x7f23c00039b0 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e0c00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00bb20 tx=0x7f23cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc043020 con 0x7f23c00039b0 2024-05-27T12:36:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc03f680 con 0x7f23c00039b0 2024-05-27T12:36:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00b600 con 0x7f23c00039b0 2024-05-27T12:36:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc0b4980 con 0x7f23c00039b0 2024-05-27T12:36:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.135+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc140250 con 0x7f23c00039b0 2024-05-27T12:36:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.136+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc075460 con 0x7f23c00039b0 2024-05-27T12:36:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.137+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23cc046020 con 0x7f23c00039b0 2024-05-27T12:36:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.138+0000 7f23d8c00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 0x7f23bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.138+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f23cc0ac310 con 0x7f23c00039b0 2024-05-27T12:36:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.139+0000 7f23dbe00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 0x7f23bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.140+0000 7f23dbe00640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 0x7f23bc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001eb0 tx=0x7f23c8001da0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.142+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23cc04d030 con 0x7f23c00039b0 2024-05-27T12:36:03.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.332+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23dc072e00 con 0x7f23c00039b0 2024-05-27T12:36:03.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.334+0000 7f23d8c00640 1 -- 172.21.15.94:0/2623417194 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12843 (secure 0 0 0) 0x7f23cc07aa80 con 0x7f23c00039b0 2024-05-27T12:36:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":29,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:35:59.340658+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 msgr2=0x7f23bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 0x7f23bc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23c8001eb0 tx=0x7f23c8001da0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 msgr2=0x7f23dc140900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00bb20 tx=0x7f23cc004730 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 shutdown_connections 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f23bc06c8f0 0x7f23bc06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23dc0b4500 0x7f23dc15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23dc0af130 0x7f23dc0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 --2- 172.21.15.94:0/2623417194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23c00039b0 0x7f23dc140900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 >> 172.21.15.94:0/2623417194 conn(0x7f23dc0b2160 msgr2=0x7f23dc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:03.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 shutdown_connections 2024-05-27T12:36:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.337+0000 7f23e2000640 1 -- 172.21.15.94:0/2623417194 wait complete. 2024-05-27T12:36:03.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:36:03.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:36:03.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.560+0000 7f9a46e00640 1 Processor -- start 2024-05-27T12:36:03.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.560+0000 7f9a46e00640 1 -- start start 2024-05-27T12:36:03.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a46e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400b4710 0x7f9a401624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a46e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9a400adb10 0x7f9a40162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a46e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4015ed10 con 0x7f9a4015e930 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a46e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9a40072930 con 0x7f9a400adb10 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a46e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9a40072000 con 0x7f9a400b4710 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a44e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9a400adb10 0x7f9a40162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a44e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9a400adb10 0x7f9a40162a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47964/0 (socket says 172.21.15.94:47964) 2024-05-27T12:36:03.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.562+0000 7f9a44e00640 1 -- 172.21.15.94:0/2185001759 learned_addr learned my addr 172.21.15.94:0/2185001759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.563+0000 7f9a3f400640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400b4710 0x7f9a401624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.563+0000 7f9a44e00640 1 -- 172.21.15.94:0/2185001759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400b4710 msgr2=0x7f9a401624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.563+0000 7f9a44e00640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400b4710 0x7f9a401624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.563+0000 7f9a44e00640 1 -- 172.21.15.94:0/2185001759 >> v1:172.21.15.94:6789/0 conn(0x7f9a4015e930 legacy=0x7f9a400b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.563+0000 7f9a44e00640 1 -- 172.21.15.94:0/2185001759 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a400722e0 con 0x7f9a400adb10 2024-05-27T12:36:03.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.564+0000 7f9a44e00640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a400adb10 0x7f9a40162a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009e50 tx=0x7f9a3802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3836b6503f46f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.564+0000 7f9a3ea00640 1 -- 172.21.15.94:0/2185001759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38036070 con 0x7f9a400adb10 2024-05-27T12:36:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.564+0000 7f9a3ea00640 1 -- 172.21.15.94:0/2185001759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a38031600 con 0x7f9a400adb10 2024-05-27T12:36:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.564+0000 7f9a3ea00640 1 -- 172.21.15.94:0/2185001759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a380319d0 con 0x7f9a400adb10 2024-05-27T12:36:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 -- 172.21.15.94:0/2185001759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a400adb10 msgr2=0x7f9a40162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a400adb10 0x7f9a40162a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009e50 tx=0x7f9a3802a710 comp rx=0 tx=0).stop 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 -- 172.21.15.94:0/2185001759 shutdown_connections 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a400adb10 0x7f9a40162a00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 --2- 172.21.15.94:0/2185001759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400b4710 0x7f9a401624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 -- 172.21.15.94:0/2185001759 >> 172.21.15.94:0/2185001759 conn(0x7f9a400b2240 msgr2=0x7f9a400b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 -- 172.21.15.94:0/2185001759 shutdown_connections 2024-05-27T12:36:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.565+0000 7f9a46e00640 1 -- 172.21.15.94:0/2185001759 wait complete. 2024-05-27T12:36:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.567+0000 7f9a46e00640 1 Processor -- start 2024-05-27T12:36:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.567+0000 7f9a46e00640 1 -- start start 2024-05-27T12:36:03.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 0x7f9a4014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 0x7f9a4015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400b3850 con 0x7f9a4015e930 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400b36d0 con 0x7f9a400b4710 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a46e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4006fd20 con 0x7f9a400adb10 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47978/0 (socket says 172.21.15.94:47978) 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 -- 172.21.15.94:0/3853124258 learned_addr learned my addr 172.21.15.94:0/3853124258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a3f400640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 0x7f9a4015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a3fe00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 0x7f9a4014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 -- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 msgr2=0x7f9a4014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 0x7f9a4014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 -- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 msgr2=0x7f9a4015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 0x7f9a4015cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a44e00640 1 -- 172.21.15.94:0/3853124258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401e7b50 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a3fe00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 0x7f9a4014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.568+0000 7f9a3f400640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 0x7f9a4015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a44e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009f70 tx=0x7f9a38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3803f020 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a38037040 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401e7e30 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38036070 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.569+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401e83f0 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.570+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40068a70 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.571+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a38042020 con 0x7f9a4015e930 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.572+0000 7f9a37e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 0x7f9a2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.572+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9a380abd00 con 0x7f9a4015e930 2024-05-27T12:36:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.573+0000 7f9a3fe00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 0x7f9a2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.574+0000 7f9a3fe00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 0x7f9a2006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a300015e0 tx=0x7f9a30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.576+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a38049030 con 0x7f9a4015e930 2024-05-27T12:36:03.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.776+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a4006e5d0 con 0x7f9a4015e930 2024-05-27T12:36:03.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.776+0000 7f9a37e00640 1 -- 172.21.15.94:0/3853124258 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f9a38079da0 con 0x7f9a4015e930 2024-05-27T12:36:03.776 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:36:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 msgr2=0x7f9a2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 0x7f9a2006ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a300015e0 tx=0x7f9a30009040 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 msgr2=0x7f9a4015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009f70 tx=0x7f9a38004a40 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 shutdown_connections 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9a2006c920 0x7f9a2006ede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a400b4710 0x7f9a4015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a4015e930 0x7f9a4015d420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 --2- 172.21.15.94:0/3853124258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a400adb10 0x7f9a4014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 >> 172.21.15.94:0/3853124258 conn(0x7f9a400b2240 msgr2=0x7f9a400b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 shutdown_connections 2024-05-27T12:36:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.779+0000 7f9a46e00640 1 -- 172.21.15.94:0/3853124258 wait complete. 2024-05-27T12:36:03.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-27T12:36:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.997+0000 7f8c97200640 1 Processor -- start 2024-05-27T12:36:03.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.997+0000 7f8c97200640 1 -- start start 2024-05-27T12:36:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c97200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c900b3460 0x7f8c900b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c97200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c900aeb20 con 0x7f8c900af5c0 2024-05-27T12:36:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c97200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c901451c0 con 0x7f8c90144de0 2024-05-27T12:36:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c97200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8c900af130 con 0x7f8c900b3460 2024-05-27T12:36:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c96800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8c900af5c0 0x7f8c90071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33000/0 (socket says 172.21.15.94:33000) 2024-05-27T12:36:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c96800640 1 -- 172.21.15.94:0/3393750526 learned_addr learned my addr 172.21.15.94:0/3393750526 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c95400640 1 --2- 172.21.15.94:0/3393750526 >> v2:172.21.15.97:3300/0 conn(0x7f8c900b3460 0x7f8c900b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:03.999+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1245819421 0 0) 0x7f8c900aeb20 con 0x7f8c900af5c0 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c78003520 con 0x7f8c900af5c0 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 229472943 0 0) 0x7f8c78003520 con 0x7f8c900af5c0 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 >> v2:172.21.15.97:3300/0 conn(0x7f8c900b3460 msgr2=0x7f8c900b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 --2- 172.21.15.94:0/3393750526 >> v2:172.21.15.97:3300/0 conn(0x7f8c900b3460 0x7f8c900b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 >> v1:172.21.15.97:6789/0 conn(0x7f8c90144de0 legacy=0x7f8c9006ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9006e880 con 0x7f8c900af5c0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f8c8c001a10 con 0x7f8c900af5c0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 >> v1:172.21.15.94:6789/0 conn(0x7f8c900af5c0 legacy=0x7f8c90071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c780039b0 0x7f8c78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c78004800 0x7f8c78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900af130 con 0x7f8c780051c0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7800c060 con 0x7f8c78004800 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.000+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7800c260 con 0x7f8c780039b0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c95400640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c78004800 0x7f8c78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c95e00640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c780039b0 0x7f8c78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 -- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c780039b0 msgr2=0x7f8c78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c780039b0 0x7f8c78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 -- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c78004800 msgr2=0x7f8c78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c78004800 0x7f8c78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.001+0000 7f8c96800640 1 -- 172.21.15.94:0/3393750526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c78003520 con 0x7f8c780051c0 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.002+0000 7f8c96800640 1 -- 172.21.15.94:0/3393750526 reap_dead start 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.002+0000 7f8c96800640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c7800b9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00a850 tx=0x7f8c78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c99e888067f8a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00a4a0 con 0x7f8c780051c0 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8c02f070 con 0x7f8c780051c0 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c94a00640 1 -- 172.21.15.94:0/3393750526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c02bcb0 con 0x7f8c780051c0 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c97200640 1 -- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 msgr2=0x7f8c7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c97200640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c7800b9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00a850 tx=0x7f8c78005830 comp rx=0 tx=0).stop 2024-05-27T12:36:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c97200640 1 -- 172.21.15.94:0/3393750526 shutdown_connections 2024-05-27T12:36:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c97200640 1 --2- 172.21.15.94:0/3393750526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c7800b9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.003+0000 7f8c97200640 1 -- 172.21.15.94:0/3393750526 >> 172.21.15.94:0/3393750526 conn(0x7f8c900b2160 msgr2=0x7f8c900b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.004+0000 7f8c97200640 1 -- 172.21.15.94:0/3393750526 shutdown_connections 2024-05-27T12:36:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.004+0000 7f8c97200640 1 -- 172.21.15.94:0/3393750526 wait complete. 2024-05-27T12:36:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.005+0000 7f8c97200640 1 Processor -- start 2024-05-27T12:36:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 -- start start 2024-05-27T12:36:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9012dcb0 0x7f8c900cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c9012f310 0x7f8c901427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9006f3c0 con 0x7f8c780051c0 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900aedc0 con 0x7f8c9012dcb0 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.006+0000 7f8c97200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8c900b38f0 con 0x7f8c9012f310 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48006/0 (socket says 172.21.15.94:48006) 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 -- 172.21.15.94:0/148861386 learned_addr learned my addr 172.21.15.94:0/148861386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95400640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9012dcb0 0x7f8c900cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 -- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c9012f310 msgr2=0x7f8c901427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c9012f310 0x7f8c901427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 -- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9012dcb0 msgr2=0x7f8c900cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9012dcb0 0x7f8c900cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 -- 172.21.15.94:0/148861386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c90130c90 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.007+0000 7f8c95e00640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e4d0 tx=0x7f8c8400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.008+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840052e0 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.008+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c84018070 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.008+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84003810 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.008+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9012e270 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.008+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9012e830 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.009+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c90075460 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.010+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c840039b0 con 0x7f8c780051c0 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.011+0000 7f8c8a000640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 0x7f8c7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.011+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8c8408b570 con 0x7f8c780051c0 2024-05-27T12:36:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.012+0000 7f8c95400640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 0x7f8c7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.013+0000 7f8c95400640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 0x7f8c7406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c90141620 tx=0x7f8c80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.015+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c90130c90 con 0x7f8c780051c0 2024-05-27T12:36:04.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.216+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8c900afb40 con 0x7f8c780051c0 2024-05-27T12:36:04.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.217+0000 7f8c8a000640 1 -- 172.21.15.94:0/148861386 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f8c84059c50 con 0x7f8c780051c0 2024-05-27T12:36:04.216 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:36:04.217 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:36:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 msgr2=0x7f8c7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 0x7f8c7406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c90141620 tx=0x7f8c80008040 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 msgr2=0x7f8c90130750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e4d0 tx=0x7f8c8400e990 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 shutdown_connections 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8c7406c920 0x7f8c7406ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c9012dcb0 0x7f8c900cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8c9012f310 0x7f8c901427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 --2- 172.21.15.94:0/148861386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c780051c0 0x7f8c90130750 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 >> 172.21.15.94:0/148861386 conn(0x7f8c900b2160 msgr2=0x7f8c900cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 shutdown_connections 2024-05-27T12:36:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.220+0000 7f8c97200640 1 -- 172.21.15.94:0/148861386 wait complete. 2024-05-27T12:36:04.238 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T12:36:04.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T12:36:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.449+0000 7f6213e00640 1 Processor -- start 2024-05-27T12:36:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.449+0000 7f6213e00640 1 -- start start 2024-05-27T12:36:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.450+0000 7f6213e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6214160de0 0x7f6214071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.450+0000 7f6213e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6214072690 con 0x7f6214072280 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.450+0000 7f6213e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62140cdf10 con 0x7f62140cdb30 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.450+0000 7f6213e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f62140ce870 con 0x7f6214160de0 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f620a000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6214160de0 0x7f6214071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f6213400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6214072280 0x7f621415d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33014/0 (socket says 172.21.15.94:33014) 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f6213400640 1 -- 172.21.15.94:0/1016607064 learned_addr learned my addr 172.21.15.94:0/1016607064 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f620a000640 1 -- 172.21.15.94:0/1016607064 >> v1:172.21.15.97:6789/0 conn(0x7f62140cdb30 legacy=0x7f62140ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f620a000640 1 -- 172.21.15.94:0/1016607064 >> v1:172.21.15.94:6789/0 conn(0x7f6214072280 legacy=0x7f621415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.451+0000 7f620a000640 1 -- 172.21.15.94:0/1016607064 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62140ceb40 con 0x7f6214160de0 2024-05-27T12:36:04.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.452+0000 7f620a000640 1 --2- 172.21.15.94:0/1016607064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6214160de0 0x7f6214071d40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0097a0 tx=0x7f61fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb9530bd714ec7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6212000640 1 -- 172.21.15.94:0/1016607064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc037070 con 0x7f6214160de0 2024-05-27T12:36:04.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6212000640 1 -- 172.21.15.94:0/1016607064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc032440 con 0x7f6214160de0 2024-05-27T12:36:04.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6212000640 1 -- 172.21.15.94:0/1016607064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc032810 con 0x7f6214160de0 2024-05-27T12:36:04.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 -- 172.21.15.94:0/1016607064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6214160de0 msgr2=0x7f6214071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 --2- 172.21.15.94:0/1016607064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6214160de0 0x7f6214071d40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0097a0 tx=0x7f61fc02a1e0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 -- 172.21.15.94:0/1016607064 shutdown_connections 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 --2- 172.21.15.94:0/1016607064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6214160de0 0x7f6214071d40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 -- 172.21.15.94:0/1016607064 >> 172.21.15.94:0/1016607064 conn(0x7f6214067e70 msgr2=0x7f621406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 -- 172.21.15.94:0/1016607064 shutdown_connections 2024-05-27T12:36:04.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.453+0000 7f6213e00640 1 -- 172.21.15.94:0/1016607064 wait complete. 2024-05-27T12:36:04.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.455+0000 7f6213e00640 1 Processor -- start 2024-05-27T12:36:04.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.455+0000 7f6213e00640 1 -- start start 2024-05-27T12:36:04.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6214072280 0x7f621413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6214160de0 0x7f6214152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62141608c0 con 0x7f62140cdb30 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f62141605e0 con 0x7f6214072280 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.456+0000 7f6213e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f621415dfc0 con 0x7f6214160de0 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48024/0 (socket says 172.21.15.94:48024) 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 -- 172.21.15.94:0/4226754473 learned_addr learned my addr 172.21.15.94:0/4226754473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f6212a00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6214072280 0x7f621413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f6213400640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6214160de0 0x7f6214152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 -- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6214160de0 msgr2=0x7f6214152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6214160de0 0x7f6214152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 -- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6214072280 msgr2=0x7f621413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:04.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6214072280 0x7f621413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:04.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.457+0000 7f620a000640 1 -- 172.21.15.94:0/4226754473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621414a550 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f620a000640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0098c0 tx=0x7f61fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc037040 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc042070 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc030a70 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6214151e10 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.458+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62141523d0 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.459+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6214075460 con 0x7f62140cdb30 2024-05-27T12:36:04.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.460+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61fc030c10 con 0x7f62140cdb30 2024-05-27T12:36:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.461+0000 7f620b400640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:04.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.461+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f61fc0ac250 con 0x7f62140cdb30 2024-05-27T12:36:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.462+0000 7f6212a00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.463+0000 7f6212a00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 0x7f61f006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6204004ea0 tx=0x7f6204009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.466+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f621414a550 con 0x7f62140cdb30 2024-05-27T12:36:04.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:04.662+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6214072d70 con 0x7f62140cdb30 2024-05-27T12:36:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.345+0000 7f620b400640 1 -- 172.21.15.94:0/4226754473 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f61fc045360 con 0x7f62140cdb30 2024-05-27T12:36:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-05-27T12:36:05.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.349+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 msgr2=0x7f61f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:05.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.349+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 0x7f61f006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6204004ea0 tx=0x7f6204009040 comp rx=0 tx=0).stop 2024-05-27T12:36:05.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.349+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 msgr2=0x7f6214150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.349+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0098c0 tx=0x7f61fc0046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 shutdown_connections 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f61f006c8f0 0x7f61f006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6214160de0 0x7f6214152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62140cdb30 0x7f6214150900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 --2- 172.21.15.94:0/4226754473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6214072280 0x7f621413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 >> 172.21.15.94:0/4226754473 conn(0x7f6214067e70 msgr2=0x7f6214074bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 shutdown_connections 2024-05-27T12:36:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.350+0000 7f6213e00640 1 -- 172.21.15.94:0/4226754473 wait complete. 2024-05-27T12:36:05.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T12:36:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.563+0000 7f9152e00640 1 Processor -- start 2024-05-27T12:36:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.564+0000 7f9152e00640 1 -- start start 2024-05-27T12:36:05.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f914c0b68e0 0x7f914c0d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 0x7f914c06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f914c0b6d40 con 0x7f914c0b68e0 2024-05-27T12:36:05.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f914c0d3690 con 0x7f914c0d3200 2024-05-27T12:36:05.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f9152e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f914c15f1b0 con 0x7f914c15edd0 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.565+0000 7f914b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f914c0b68e0 0x7f914c0d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f914c0b68e0 0x7f914c0d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48032/0 (socket says 172.21.15.94:48032) 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 -- 172.21.15.94:0/742431930 learned_addr learned my addr 172.21.15.94:0/742431930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914be00640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f9150e00640 1 --2- 172.21.15.94:0/742431930 >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 0x7f914c06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 -- 172.21.15.94:0/742431930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 msgr2=0x7f914c0b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:05.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 -- 172.21.15.94:0/742431930 >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 msgr2=0x7f914c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:05.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 --2- 172.21.15.94:0/742431930 >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 0x7f914c06ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f914b400640 1 -- 172.21.15.94:0/742431930 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0d3940 con 0x7f914c0b68e0 2024-05-27T12:36:05.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.566+0000 7f9150e00640 1 --2- 172.21.15.94:0/742431930 >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 0x7f914c06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:05.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.567+0000 7f914b400640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0b68e0 0x7f914c0d2cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f914c0d2580 tx=0x7f913402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b29a37568cb977 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:05.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.568+0000 7f914aa00640 1 -- 172.21.15.94:0/742431930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134037070 con 0x7f914c0b68e0 2024-05-27T12:36:05.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.568+0000 7f914aa00640 1 -- 172.21.15.94:0/742431930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134032440 con 0x7f914c0b68e0 2024-05-27T12:36:05.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.568+0000 7f914aa00640 1 -- 172.21.15.94:0/742431930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134032810 con 0x7f914c0b68e0 2024-05-27T12:36:05.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 -- 172.21.15.94:0/742431930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0b68e0 msgr2=0x7f914c0d2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0b68e0 0x7f914c0d2cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f914c0d2580 tx=0x7f913402a1e0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 -- 172.21.15.94:0/742431930 shutdown_connections 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 --2- 172.21.15.94:0/742431930 >> v2:172.21.15.97:3300/0 conn(0x7f914c0d3200 0x7f914c06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0b68e0 0x7f914c0d2cc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 --2- 172.21.15.94:0/742431930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 -- 172.21.15.94:0/742431930 >> 172.21.15.94:0/742431930 conn(0x7f914c0d13a0 msgr2=0x7f914c0d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 -- 172.21.15.94:0/742431930 shutdown_connections 2024-05-27T12:36:05.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.569+0000 7f9152e00640 1 -- 172.21.15.94:0/742431930 wait complete. 2024-05-27T12:36:05.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.570+0000 7f9152e00640 1 Processor -- start 2024-05-27T12:36:05.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.571+0000 7f9152e00640 1 -- start start 2024-05-27T12:36:05.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914c0b68e0 0x7f914c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0b80d0 con 0x7f914c0d3200 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0d3f00 con 0x7f914c0b68e0 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f9152e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f914c0b7d30 con 0x7f914c15edd0 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48042/0 (socket says 172.21.15.94:48042) 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 -- 172.21.15.94:0/2355909488 learned_addr learned my addr 172.21.15.94:0/2355909488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914be00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914c0b68e0 0x7f914c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 -- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 msgr2=0x7f914c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 -- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914c0b68e0 msgr2=0x7f914c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914c0b68e0 0x7f914c14df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.572+0000 7f914b400640 1 -- 172.21.15.94:0/2355909488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0ac6d0 con 0x7f914c0d3200 2024-05-27T12:36:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f914b400640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f91340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134041020 con 0x7f914c0d3200 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91340306c0 con 0x7f914c0d3200 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134037040 con 0x7f914c0d3200 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c0ac9b0 con 0x7f914c0d3200 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.573+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c0acf70 con 0x7f914c0d3200 2024-05-27T12:36:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.574+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c068b50 con 0x7f914c0d3200 2024-05-27T12:36:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.575+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9134044020 con 0x7f914c0d3200 2024-05-27T12:36:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.576+0000 7f9143e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.576+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f91340ac1b0 con 0x7f914c0d3200 2024-05-27T12:36:05.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.577+0000 7f914be00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:05.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.578+0000 7f914be00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f913c006fd0 tx=0x7f913c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.582+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f913404b030 con 0x7f914c0d3200 2024-05-27T12:36:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:05.782+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f914c15f560 con 0x7f914c0d3200 2024-05-27T12:36:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.361+0000 7f9143e00640 1 -- 172.21.15.94:0/2355909488 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f913407a7b0 con 0x7f914c0d3200 2024-05-27T12:36:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-05-27T12:36:06.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.365+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 msgr2=0x7f912806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f913c006fd0 tx=0x7f913c008040 comp rx=0 tx=0).stop 2024-05-27T12:36:06.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 msgr2=0x7f914c0abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f91340046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 shutdown_connections 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f914c15edd0 0x7f914c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f914c0d3200 0x7f914c0abca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 --2- 172.21.15.94:0/2355909488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914c0b68e0 0x7f914c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.366+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 >> 172.21.15.94:0/2355909488 conn(0x7f914c0d13a0 msgr2=0x7f914c0cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.368+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 shutdown_connections 2024-05-27T12:36:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.368+0000 7f9152e00640 1 -- 172.21.15.94:0/2355909488 wait complete. 2024-05-27T12:36:06.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T12:36:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.580+0000 7f7c5cc00640 1 Processor -- start 2024-05-27T12:36:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.580+0000 7f7c5cc00640 1 -- start start 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c5cc00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c5815b700 0x7f7c580ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 0x7f7c5806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c5cc00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5815bae0 con 0x7f7c5806be70 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c5cc00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7c580cf6e0 con 0x7f7c5815b700 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c5cc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7c580b3540 con 0x7f7c5806fe70 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c55e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c5815b700 0x7f7c580ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c55e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7c5815b700 0x7f7c580ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48048/0 (socket says 172.21.15.94:48048) 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c56800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7c5806be70 0x7f7c5806de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33022/0 (socket says 172.21.15.94:33022) 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c55e00640 1 -- 172.21.15.94:0/843454513 learned_addr learned my addr 172.21.15.94:0/843454513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c55e00640 1 -- 172.21.15.94:0/843454513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 msgr2=0x7f7c5806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.582+0000 7f7c4fe00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 0x7f7c5806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.583+0000 7f7c55e00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 0x7f7c5806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.583+0000 7f7c55e00640 1 -- 172.21.15.94:0/843454513 >> v1:172.21.15.94:6789/0 conn(0x7f7c5806be70 legacy=0x7f7c5806de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.583+0000 7f7c55400640 1 -- 172.21.15.94:0/843454513 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564628831 0 0) 0x7f7c5815bae0 con 0x7f7c5806be70 2024-05-27T12:36:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.583+0000 7f7c55e00640 1 -- 172.21.15.94:0/843454513 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580b3820 con 0x7f7c5815b700 2024-05-27T12:36:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.583+0000 7f7c4fe00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 0x7f7c5806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c55e00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c580ce3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb704a1584622e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c55400640 1 -- 172.21.15.94:0/843454513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037070 con 0x7f7c5815b700 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c55400640 1 -- 172.21.15.94:0/843454513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c48032440 con 0x7f7c5815b700 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c55400640 1 -- 172.21.15.94:0/843454513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48032810 con 0x7f7c5815b700 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c5cc00640 1 -- 172.21.15.94:0/843454513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 msgr2=0x7f7c580ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.584+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c580ce3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4802a1c0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 -- 172.21.15.94:0/843454513 shutdown_connections 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806fe70 0x7f7c5806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/843454513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c580ce3b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 -- 172.21.15.94:0/843454513 >> 172.21.15.94:0/843454513 conn(0x7f7c580b2240 msgr2=0x7f7c580b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 -- 172.21.15.94:0/843454513 shutdown_connections 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.585+0000 7f7c5cc00640 1 -- 172.21.15.94:0/843454513 wait complete. 2024-05-27T12:36:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.586+0000 7f7c5cc00640 1 Processor -- start 2024-05-27T12:36:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.587+0000 7f7c5cc00640 1 -- start start 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.587+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806be70 0x7f7c58147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.587+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5806fe70 0x7f7c58157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.587+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.587+0000 7f7c5cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580cf3a0 con 0x7f7c5815b700 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c5cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580b3de0 con 0x7f7c5806fe70 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c5cc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7c580b5310 con 0x7f7c5806be70 2024-05-27T12:36:06.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48052/0 (socket says 172.21.15.94:48052) 2024-05-27T12:36:06.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 -- 172.21.15.94:0/1017360357 learned_addr learned my addr 172.21.15.94:0/1017360357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c55e00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806be70 0x7f7c58147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c4fe00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5806fe70 0x7f7c58157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 -- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806be70 msgr2=0x7f7c58147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806be70 0x7f7c58147df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 -- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5806fe70 msgr2=0x7f7c58157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5806fe70 0x7f7c58157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.588+0000 7f7c56800640 1 -- 172.21.15.94:0/1017360357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581548c0 con 0x7f7c5815b700 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c56800640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c7a0 tx=0x7f7c5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50018070 con 0x7f7c5815b700 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5000a640 con 0x7f7c5815b700 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000aab0 con 0x7f7c5815b700 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c58154ba0 con 0x7f7c5815b700 2024-05-27T12:36:06.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.589+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c58155160 con 0x7f7c5815b700 2024-05-27T12:36:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.590+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58068a70 con 0x7f7c5815b700 2024-05-27T12:36:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.591+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c5001b070 con 0x7f7c5815b700 2024-05-27T12:36:06.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.592+0000 7f7c4e000640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.592+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f7c5008b130 con 0x7f7c5815b700 2024-05-27T12:36:06.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.593+0000 7f7c55e00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.594+0000 7f7c55e00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 0x7f7c3406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c48034000 tx=0x7f7c4802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.598+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c500595b0 con 0x7f7c5815b700 2024-05-27T12:36:06.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:06.825+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c580d0f60 con 0x7f7c5815b700 2024-05-27T12:36:07.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.392+0000 7f7c4e000640 1 -- 172.21.15.94:0/1017360357 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f7c50058eb0 con 0x7f7c5815b700 2024-05-27T12:36:07.392 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 4 and data pool 5 2024-05-27T12:36:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.398+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 msgr2=0x7f7c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.398+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 0x7f7c3406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c48034000 tx=0x7f7c4802e000 comp rx=0 tx=0).stop 2024-05-27T12:36:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.398+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 msgr2=0x7f7c581575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.398+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c7a0 tx=0x7f7c5000cc60 comp rx=0 tx=0).stop 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 shutdown_connections 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5806fe70 0x7f7c58157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c5815b700 0x7f7c581575b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 --2- 172.21.15.94:0/1017360357 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c5806be70 0x7f7c58147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.399+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 >> 172.21.15.94:0/1017360357 conn(0x7f7c580b2240 msgr2=0x7f7c580b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.401+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 shutdown_connections 2024-05-27T12:36:07.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.404+0000 7f7c5cc00640 1 -- 172.21.15.94:0/1017360357 wait complete. 2024-05-27T12:36:07.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:36:07.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.595+0000 7f7626000640 1 Processor -- start 2024-05-27T12:36:07.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.596+0000 7f7626000640 1 -- start start 2024-05-27T12:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f7626000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76200b2f10 0x7f7620071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f7626000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f762015ed00 con 0x7f7620072460 2024-05-27T12:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f7626000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76200cd580 con 0x7f762015e920 2024-05-27T12:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f7626000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7620072890 con 0x7f76200b2f10 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f761fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f7620072460 0x7f76200ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33030/0 (socket says 172.21.15.94:33030) 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.597+0000 7f761fe00640 1 -- 172.21.15.94:0/731660284 learned_addr learned my addr 172.21.15.94:0/731660284 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761ea00640 1 --2- 172.21.15.94:0/731660284 >> v2:172.21.15.97:3300/0 conn(0x7f76200b2f10 0x7f7620071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745478567 0 0) 0x7f762015ed00 con 0x7f7620072460 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7604003520 con 0x7f7620072460 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761282890 0 0) 0x7f7604003520 con 0x7f7620072460 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 >> v2:172.21.15.97:3300/0 conn(0x7f76200b2f10 msgr2=0x7f7620071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 --2- 172.21.15.94:0/731660284 >> v2:172.21.15.97:3300/0 conn(0x7f76200b2f10 0x7f7620071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 >> v1:172.21.15.97:6789/0 conn(0x7f762015e920 legacy=0x7f7620162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76200cd860 con 0x7f7620072460 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f7618001a10 con 0x7f7620072460 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 >> v1:172.21.15.94:6789/0 conn(0x7f7620072460 legacy=0x7f76200ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7604003ae0 0x7f7604003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7604004800 0x7f7604004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f760400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7620072890 con 0x7f7604005230 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f760400c190 con 0x7f7604003ae0 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.598+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f760400c390 con 0x7f7604004800 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f760400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761f400640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7604003ae0 0x7f7604003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 -- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7604004800 msgr2=0x7f7604004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7604004800 0x7f7604004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 -- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7604003ae0 msgr2=0x7f7604003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7604003ae0 0x7f7604003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761fe00640 1 -- 172.21.15.94:0/731660284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7604003520 con 0x7f7604005230 2024-05-27T12:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.599+0000 7f761f400640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7604003ae0 0x7f7604003ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.600+0000 7f761fe00640 1 -- 172.21.15.94:0/731660284 reap_dead start 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.600+0000 7f761fe00640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f760400b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f761800a7a0 tx=0x7f7604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b430390e4325afb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.600+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800a3f0 con 0x7f7604005230 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.600+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761802f070 con 0x7f7604005230 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.600+0000 7f761e000640 1 -- 172.21.15.94:0/731660284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761802baf0 con 0x7f7604005230 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 -- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 msgr2=0x7f760400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f760400b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f761800a7a0 tx=0x7f7604005830 comp rx=0 tx=0).stop 2024-05-27T12:36:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 -- 172.21.15.94:0/731660284 shutdown_connections 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 --2- 172.21.15.94:0/731660284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f760400b9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 -- 172.21.15.94:0/731660284 >> 172.21.15.94:0/731660284 conn(0x7f76200b3830 msgr2=0x7f76200b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 -- 172.21.15.94:0/731660284 shutdown_connections 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.601+0000 7f7626000640 1 -- 172.21.15.94:0/731660284 wait complete. 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.603+0000 7f7626000640 1 Processor -- start 2024-05-27T12:36:07.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.603+0000 7f7626000640 1 -- start start 2024-05-27T12:36:07.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7620160860 0x7f7620159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f76201e7d60 0x7f76201e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76200ce100 con 0x7f7604005230 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76200cde20 con 0x7f7620160860 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f7626000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7620071f20 con 0x7f76201e7d60 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48064/0 (socket says 172.21.15.94:48064) 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 -- 172.21.15.94:0/2256264445 learned_addr learned my addr 172.21.15.94:0/2256264445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 -- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f76201e7d60 msgr2=0x7f76201e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f76201e7d60 0x7f76201e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 -- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7620160860 msgr2=0x7f7620159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7620160860 0x7f7620159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.604+0000 7f761f400640 1 -- 172.21.15.94:0/2256264445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7620143260 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f761f400640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f761000d5c0 tx=0x7f761000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610016070 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7610007240 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610007630 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7620160ed0 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.605+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7620161490 con 0x7f7604005230 2024-05-27T12:36:07.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.606+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7620075460 con 0x7f7604005230 2024-05-27T12:36:07.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.607+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76100067d0 con 0x7f7604005230 2024-05-27T12:36:07.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.608+0000 7f7617400640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 0x7f760006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:07.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.608+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f761008b960 con 0x7f7604005230 2024-05-27T12:36:07.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.609+0000 7f761ea00640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 0x7f760006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:07.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.610+0000 7f761ea00640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 0x7f760006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f760c006a90 tx=0x7f760c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:07.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.612+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7610059d90 con 0x7f7604005230 2024-05-27T12:36:07.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.809+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7620143430 con 0x7f7604005230 2024-05-27T12:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.811+0000 7f7617400640 1 -- 172.21.15.94:0/2256264445 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16634 (secure 0 0 0) 0x7f7610059690 con 0x7f7604005230 2024-05-27T12:36:07.810 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:07.811 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:36:06.826997+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:36:04.679045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-27T12:36:05.799460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 msgr2=0x7f760006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 0x7f760006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f760c006a90 tx=0x7f760c006a20 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 msgr2=0x7f7620142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f761000d5c0 tx=0x7f761000da80 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 shutdown_connections 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f760006c920 0x7f760006ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7620160860 0x7f7620159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f76201e7d60 0x7f76201e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 --2- 172.21.15.94:0/2256264445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7604005230 0x7f7620142bb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:07.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 >> 172.21.15.94:0/2256264445 conn(0x7f76200b3830 msgr2=0x7f76200ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:07.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.813+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 shutdown_connections 2024-05-27T12:36:07.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:07.814+0000 7f7626000640 1 -- 172.21.15.94:0/2256264445 wait complete. 2024-05-27T12:36:07.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:36:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.032+0000 7f5128800640 1 Processor -- start 2024-05-27T12:36:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.032+0000 7f5128800640 1 -- start start 2024-05-27T12:36:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5128800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f51200cf400 0x7f51200cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5128800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51200c5710 0x7f51200c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5128800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512006f450 con 0x7f512015a250 2024-05-27T12:36:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5128800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f5120160f70 con 0x7f51200cf400 2024-05-27T12:36:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5128800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f51200cfd80 con 0x7f51200c5710 2024-05-27T12:36:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5125400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f51200cf400 0x7f51200cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5125400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f51200cf400 0x7f51200cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48076/0 (socket says 172.21.15.94:48076) 2024-05-27T12:36:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5125400640 1 -- 172.21.15.94:0/1541327639 learned_addr learned my addr 172.21.15.94:0/1541327639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:08.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.034+0000 7f5126800640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51200c5710 0x7f51200c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.035+0000 7f5125400640 1 -- 172.21.15.94:0/1541327639 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51200c5710 msgr2=0x7f51200c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.035+0000 7f5125400640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51200c5710 0x7f51200c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.035+0000 7f5125400640 1 -- 172.21.15.94:0/1541327639 >> v1:172.21.15.97:6789/0 conn(0x7f512015a250 legacy=0x7f512006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.035+0000 7f5125400640 1 -- 172.21.15.94:0/1541327639 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5120160360 con 0x7f51200cf400 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.036+0000 7f5125400640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f51200cf840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5110009870 tx=0x7f511002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a1059266664e28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.036+0000 7f5124a00640 1 -- 172.21.15.94:0/1541327639 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110037070 con 0x7f51200cf400 2024-05-27T12:36:08.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.036+0000 7f5124a00640 1 -- 172.21.15.94:0/1541327639 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110032440 con 0x7f51200cf400 2024-05-27T12:36:08.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.036+0000 7f5124a00640 1 -- 172.21.15.94:0/1541327639 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110032810 con 0x7f51200cf400 2024-05-27T12:36:08.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 -- 172.21.15.94:0/1541327639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 msgr2=0x7f51200cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f51200cf840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5110009870 tx=0x7f511002a270 comp rx=0 tx=0).stop 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 -- 172.21.15.94:0/1541327639 shutdown_connections 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51200c5710 0x7f51200c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 --2- 172.21.15.94:0/1541327639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f51200cf840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 -- 172.21.15.94:0/1541327639 >> 172.21.15.94:0/1541327639 conn(0x7f51200b3830 msgr2=0x7f51200b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 -- 172.21.15.94:0/1541327639 shutdown_connections 2024-05-27T12:36:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.037+0000 7f5128800640 1 -- 172.21.15.94:0/1541327639 wait complete. 2024-05-27T12:36:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.039+0000 7f5128800640 1 Processor -- start 2024-05-27T12:36:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.039+0000 7f5128800640 1 -- start start 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51200c5710 0x7f512014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f512015a250 0x7f512015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5120160c00 con 0x7f51200cf400 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5120160920 con 0x7f51200c5710 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5128800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f512015a630 con 0x7f512015a250 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5125400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5125400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48090/0 (socket says 172.21.15.94:48090) 2024-05-27T12:36:08.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5125400640 1 -- 172.21.15.94:0/3738925649 learned_addr learned my addr 172.21.15.94:0/3738925649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.040+0000 7f5125e00640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51200c5710 0x7f512014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5126800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f512015a250 0x7f512015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 -- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f512015a250 msgr2=0x7f512015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f512015a250 0x7f512015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 -- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51200c5710 msgr2=0x7f512014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51200c5710 0x7f512014d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 -- 172.21.15.94:0/3738925649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512014e220 con 0x7f51200cf400 2024-05-27T12:36:08.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5125400640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5110000c00 tx=0x7f51100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110041020 con 0x7f51200cf400 2024-05-27T12:36:08.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51100306c0 con 0x7f51200cf400 2024-05-27T12:36:08.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110037040 con 0x7f51200cf400 2024-05-27T12:36:08.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.041+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512014e500 con 0x7f51200cf400 2024-05-27T12:36:08.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.042+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512014eac0 con 0x7f51200cf400 2024-05-27T12:36:08.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.043+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5120075460 con 0x7f51200cf400 2024-05-27T12:36:08.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.044+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5110044020 con 0x7f51200cf400 2024-05-27T12:36:08.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.045+0000 7f511a000640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 0x7f510406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.045+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f51100ac330 con 0x7f51200cf400 2024-05-27T12:36:08.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.045+0000 7f5125e00640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 0x7f510406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.047+0000 7f5125e00640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 0x7f510406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114000f60 tx=0x7f5114009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.049+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f511004b030 con 0x7f51200cf400 2024-05-27T12:36:08.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.250+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51200c6090 con 0x7f51200cf400 2024-05-27T12:36:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.251+0000 7f511a000640 1 -- 172.21.15.94:0/3738925649 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16634 (secure 0 0 0) 0x7f511007a760 con 0x7f51200cf400 2024-05-27T12:36:08.251 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:08.251 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:36:06.826997+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:36:04.679045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-27T12:36:05.799460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 msgr2=0x7f510406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 0x7f510406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114000f60 tx=0x7f5114009040 comp rx=0 tx=0).stop 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 msgr2=0x7f512014db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5110000c00 tx=0x7f51100046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 shutdown_connections 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f510406c8f0 0x7f510406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f512015a250 0x7f512015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51200cf400 0x7f512014db90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 --2- 172.21.15.94:0/3738925649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51200c5710 0x7f512014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.254+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 >> 172.21.15.94:0/3738925649 conn(0x7f51200b3830 msgr2=0x7f512015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.255+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 shutdown_connections 2024-05-27T12:36:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.255+0000 7f5128800640 1 -- 172.21.15.94:0/3738925649 wait complete. 2024-05-27T12:36:08.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-27T12:36:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.478+0000 7f332d600640 1 Processor -- start 2024-05-27T12:36:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.478+0000 7f332d600640 1 -- start start 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.480+0000 7f332d600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f332815e930 0x7f33280b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.480+0000 7f332d600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33280adb10 0x7f33280b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.480+0000 7f332d600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332815ed10 con 0x7f332806fd20 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.480+0000 7f332d600640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3328072930 con 0x7f332815e930 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.480+0000 7f332d600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3328072000 con 0x7f33280adb10 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3326a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f332815e930 0x7f33280b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3327400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f332806fd20 0x7f332806eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33046/0 (socket says 172.21.15.94:33046) 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3327400640 1 -- 172.21.15.94:0/1754987636 learned_addr learned my addr 172.21.15.94:0/1754987636 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f331e000640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33280adb10 0x7f33280b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3326a00640 1 -- 172.21.15.94:0/1754987636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33280adb10 msgr2=0x7f33280b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3326a00640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33280adb10 0x7f33280b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3326a00640 1 -- 172.21.15.94:0/1754987636 >> v1:172.21.15.94:6789/0 conn(0x7f332806fd20 legacy=0x7f332806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.481+0000 7f3326a00640 1 -- 172.21.15.94:0/1754987636 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33280722e0 con 0x7f332815e930 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.482+0000 7f3326a00640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332815e930 0x7f33280b4000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f33180097a0 tx=0x7f331802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a021e9e9a4999e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.482+0000 7f3326000640 1 -- 172.21.15.94:0/1754987636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318037070 con 0x7f332815e930 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.482+0000 7f3326000640 1 -- 172.21.15.94:0/1754987636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3318032440 con 0x7f332815e930 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.482+0000 7f3326000640 1 -- 172.21.15.94:0/1754987636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318032810 con 0x7f332815e930 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 -- 172.21.15.94:0/1754987636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332815e930 msgr2=0x7f33280b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332815e930 0x7f33280b4000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f33180097a0 tx=0x7f331802a180 comp rx=0 tx=0).stop 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 -- 172.21.15.94:0/1754987636 shutdown_connections 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33280adb10 0x7f33280b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 --2- 172.21.15.94:0/1754987636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f332815e930 0x7f33280b4000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 -- 172.21.15.94:0/1754987636 >> 172.21.15.94:0/1754987636 conn(0x7f33280b2240 msgr2=0x7f33280b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 -- 172.21.15.94:0/1754987636 shutdown_connections 2024-05-27T12:36:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.483+0000 7f332d600640 1 -- 172.21.15.94:0/1754987636 wait complete. 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.485+0000 7f332d600640 1 Processor -- start 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.485+0000 7f332d600640 1 -- start start 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 0x7f3328157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332815e930 0x7f33280d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33280b3850 con 0x7f33280adb10 2024-05-27T12:36:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33280b36d0 con 0x7f332815e930 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f332d600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f33280afbf0 con 0x7f332806fd20 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f331e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f331e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48104/0 (socket says 172.21.15.94:48104) 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.486+0000 7f331e000640 1 -- 172.21.15.94:0/2569595719 learned_addr learned my addr 172.21.15.94:0/2569595719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f3326a00640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 0x7f3328157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 -- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 msgr2=0x7f3328157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 0x7f3328157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 -- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332815e930 msgr2=0x7f33280d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332815e930 0x7f33280d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 -- 172.21.15.94:0/2569595719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281f8af0 con 0x7f33280adb10 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f3326a00640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 0x7f3328157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331e000640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f331000d800 tx=0x7f331000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331000b6c0 con 0x7f33280adb10 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3310019070 con 0x7f33280adb10 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331000ba80 con 0x7f33280adb10 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.487+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3328157e20 con 0x7f33280adb10 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.488+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281583e0 con 0x7f33280adb10 2024-05-27T12:36:08.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.489+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328068a70 con 0x7f33280adb10 2024-05-27T12:36:08.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.489+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f331000bc20 con 0x7f33280adb10 2024-05-27T12:36:08.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.490+0000 7f331f400640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 0x7f330406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:08.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.490+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f331008bff0 con 0x7f33280adb10 2024-05-27T12:36:08.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.491+0000 7f3326a00640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 0x7f330406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:08.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.492+0000 7f3326a00640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 0x7f330406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3318034000 tx=0x7f331802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:08.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.495+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33281f8af0 con 0x7f33280adb10 2024-05-27T12:36:08.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:08.736+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f332806e5d0 con 0x7f33280adb10 2024-05-27T12:36:09.334 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-05-27T12:36:09.335+0000 7fd71fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T12:36:09.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.404+0000 7f331f400640 1 -- 172.21.15.94:0/2569595719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f331005a3e0 con 0x7f33280adb10 2024-05-27T12:36:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.408+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 msgr2=0x7f330406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.408+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 0x7f330406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3318034000 tx=0x7f331802e000 comp rx=0 tx=0).stop 2024-05-27T12:36:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.408+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 msgr2=0x7f33281f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.408+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f331000d800 tx=0x7f331000dcc0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 shutdown_connections 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f330406c8f0 0x7f330406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332815e930 0x7f33280d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280adb10 0x7f33281f85b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 --2- 172.21.15.94:0/2569595719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f332806fd20 0x7f3328157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 >> 172.21.15.94:0/2569595719 conn(0x7f33280b2240 msgr2=0x7f33280adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 shutdown_connections 2024-05-27T12:36:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.409+0000 7f332d600640 1 -- 172.21.15.94:0/2569595719 wait complete. 2024-05-27T12:36:09.433 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-27T12:36:09.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-27T12:36:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.633+0000 7f3bbd600640 1 Processor -- start 2024-05-27T12:36:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.634+0000 7f3bbd600640 1 -- start start 2024-05-27T12:36:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bbd600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f3bb80d4320 0x7f3bb81697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bbd600640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80b7840 con 0x7f3bb815e900 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bbd600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb8067f30 con 0x7f3bb80ba3f0 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bbd600640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb8068210 con 0x7f3bb80d4320 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bb7e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f3bb815e900 0x7f3bb8067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:33052/0 (socket says 172.21.15.94:33052) 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bb7e00640 1 -- 172.21.15.94:0/866358635 learned_addr learned my addr 172.21.15.94:0/866358635 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.635+0000 7f3bbcc00640 1 --2- 172.21.15.94:0/866358635 >> v2:172.21.15.94:3300/0 conn(0x7f3bb80d4320 0x7f3bb81697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.636+0000 7f3bbcc00640 1 -- 172.21.15.94:0/866358635 >> v1:172.21.15.97:6789/0 conn(0x7f3bb80ba3f0 legacy=0x7f3bb80b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.636+0000 7f3bbcc00640 1 -- 172.21.15.94:0/866358635 >> v1:172.21.15.94:6789/0 conn(0x7f3bb815e900 legacy=0x7f3bb8067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.636+0000 7f3bbcc00640 1 -- 172.21.15.94:0/866358635 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80684f0 con 0x7f3bb80d4320 2024-05-27T12:36:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.637+0000 7f3bbcc00640 1 --2- 172.21.15.94:0/866358635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb80d4320 0x7f3bb81697b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00097a0 tx=0x7f3bb002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144a64c6ef134cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.637+0000 7f3bb6a00640 1 -- 172.21.15.94:0/866358635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0037070 con 0x7f3bb80d4320 2024-05-27T12:36:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.637+0000 7f3bb6a00640 1 -- 172.21.15.94:0/866358635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb0032640 con 0x7f3bb80d4320 2024-05-27T12:36:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.637+0000 7f3bb6a00640 1 -- 172.21.15.94:0/866358635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0032a10 con 0x7f3bb80d4320 2024-05-27T12:36:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 -- 172.21.15.94:0/866358635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb80d4320 msgr2=0x7f3bb81697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 --2- 172.21.15.94:0/866358635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb80d4320 0x7f3bb81697b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00097a0 tx=0x7f3bb002a2c0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 -- 172.21.15.94:0/866358635 shutdown_connections 2024-05-27T12:36:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 --2- 172.21.15.94:0/866358635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb80d4320 0x7f3bb81697b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 -- 172.21.15.94:0/866358635 >> 172.21.15.94:0/866358635 conn(0x7f3bb80d1190 msgr2=0x7f3bb80d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 -- 172.21.15.94:0/866358635 shutdown_connections 2024-05-27T12:36:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.638+0000 7f3bbd600640 1 -- 172.21.15.94:0/866358635 wait complete. 2024-05-27T12:36:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.640+0000 7f3bbd600640 1 Processor -- start 2024-05-27T12:36:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.640+0000 7f3bbd600640 1 -- start start 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bb80ba3f0 0x7f3bb80cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb8159f00 0x7f3bb8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb815ece0 con 0x7f3bb815e900 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80d2df0 con 0x7f3bb8159f00 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.641+0000 7f3bbd600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb80b68a0 con 0x7f3bb80ba3f0 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48128/0 (socket says 172.21.15.94:48128) 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 -- 172.21.15.94:0/1578779069 learned_addr learned my addr 172.21.15.94:0/1578779069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bbcc00640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb8159f00 0x7f3bb8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7e00640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bb80ba3f0 0x7f3bb80cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 -- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bb80ba3f0 msgr2=0x7f3bb80cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bb80ba3f0 0x7f3bb80cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 -- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb8159f00 msgr2=0x7f3bb8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb8159f00 0x7f3bb8160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.642+0000 7f3bb7400640 1 -- 172.21.15.94:0/1578779069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb8161870 con 0x7f3bb815e900 2024-05-27T12:36:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bb7400640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c790 tx=0x7f3ba000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0016020 con 0x7f3bb815e900 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0006d50 con 0x7f3bb815e900 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb8161b50 con 0x7f3bb815e900 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003bb0 con 0x7f3bb815e900 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.643+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8144b10 con 0x7f3bb815e900 2024-05-27T12:36:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.645+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb815a310 con 0x7f3bb815e900 2024-05-27T12:36:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.645+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ba000a850 con 0x7f3bb815e900 2024-05-27T12:36:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.646+0000 7f3bafe00640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.647+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3ba008b8a0 con 0x7f3bb815e900 2024-05-27T12:36:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.647+0000 7f3bb7e00640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.648+0000 7f3bb7e00640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 0x7f3b9406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8005880 tx=0x7f3ba8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.651+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bb815a310 con 0x7f3bb815e900 2024-05-27T12:36:09.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:09.892+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3bb80b6570 con 0x7f3bb815e900 2024-05-27T12:36:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.416+0000 7f3bafe00640 1 -- 172.21.15.94:0/1578779069 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f3ba0020020 con 0x7f3bb815e900 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 msgr2=0x7f3b9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 0x7f3b9406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8005880 tx=0x7f3ba8009040 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 msgr2=0x7f3bb81599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c790 tx=0x7f3ba000cc50 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 shutdown_connections 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f3b9406c8f0 0x7f3b9406edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb8159f00 0x7f3bb8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb815e900 0x7f3bb81599c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 --2- 172.21.15.94:0/1578779069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bb80ba3f0 0x7f3bb80cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 >> 172.21.15.94:0/1578779069 conn(0x7f3bb80d1190 msgr2=0x7f3bb80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:10.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 shutdown_connections 2024-05-27T12:36:10.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.420+0000 7f3bbd600640 1 -- 172.21.15.94:0/1578779069 wait complete. 2024-05-27T12:36:10.442 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-27T12:36:10.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:36:10.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.642+0000 7f9668600640 1 Processor -- start 2024-05-27T12:36:10.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.642+0000 7f9668600640 1 -- start start 2024-05-27T12:36:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.643+0000 7f9668600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 0x7f96600cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.643+0000 7f9668600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9668600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600ce200 con 0x7f96600cd6c0 2024-05-27T12:36:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9668600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f9660144f90 con 0x7f9660072630 2024-05-27T12:36:10.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9668600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f96600cdc40 con 0x7f9660144bb0 2024-05-27T12:36:10.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9666600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:10.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9666600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:40170/0 (socket says 172.21.15.94:40170) 2024-05-27T12:36:10.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9666600640 1 -- 172.21.15.94:0/4195426909 learned_addr learned my addr 172.21.15.94:0/4195426909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:10.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.644+0000 7f9665c00640 1 --2- 172.21.15.94:0/4195426909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 0x7f96600cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:10.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.645+0000 7f9665c00640 1 -- 172.21.15.94:0/4195426909 >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 msgr2=0x7f96600cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:10.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.645+0000 7f9665c00640 1 --2- 172.21.15.94:0/4195426909 >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.645+0000 7f9665c00640 1 -- 172.21.15.94:0/4195426909 >> v1:172.21.15.97:6789/0 conn(0x7f96600cd6c0 legacy=0x7f9660071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.645+0000 7f9665c00640 1 -- 172.21.15.94:0/4195426909 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600cdf20 con 0x7f9660144bb0 2024-05-27T12:36:10.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.645+0000 7f9666600640 1 --2- 172.21.15.94:0/4195426909 >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T12:36:10.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.647+0000 7f9665c00640 1 --2- 172.21.15.94:0/4195426909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 0x7f96600cd180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9654009d90 tx=0x7f965402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f69530619eec5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:10.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f965fe00640 1 -- 172.21.15.94:0/4195426909 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654007740 con 0x7f9660144bb0 2024-05-27T12:36:10.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f965fe00640 1 -- 172.21.15.94:0/4195426909 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96540078e0 con 0x7f9660144bb0 2024-05-27T12:36:10.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f965fe00640 1 -- 172.21.15.94:0/4195426909 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654007cb0 con 0x7f9660144bb0 2024-05-27T12:36:10.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f9668600640 1 -- 172.21.15.94:0/4195426909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 msgr2=0x7f96600cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f9668600640 1 --2- 172.21.15.94:0/4195426909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 0x7f96600cd180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9654009d90 tx=0x7f965402a660 comp rx=0 tx=0).stop 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.648+0000 7f9668600640 1 -- 172.21.15.94:0/4195426909 shutdown_connections 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.649+0000 7f9668600640 1 --2- 172.21.15.94:0/4195426909 >> v2:172.21.15.97:3300/0 conn(0x7f9660072630 0x7f96600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.649+0000 7f9668600640 1 --2- 172.21.15.94:0/4195426909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9660144bb0 0x7f96600cd180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.649+0000 7f9668600640 1 -- 172.21.15.94:0/4195426909 >> 172.21.15.94:0/4195426909 conn(0x7f96600b3830 msgr2=0x7f96600b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.649+0000 7f9668600640 1 -- 172.21.15.94:0/4195426909 shutdown_connections 2024-05-27T12:36:10.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.649+0000 7f9668600640 1 -- 172.21.15.94:0/4195426909 wait complete. 2024-05-27T12:36:10.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.650+0000 7f9668600640 1 Processor -- start 2024-05-27T12:36:10.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.651+0000 7f9668600640 1 -- start start 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96600cd6c0 0x7f966006df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9660144bb0 0x7f966006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9660145330 con 0x7f9660072630 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96600ceb20 con 0x7f9660144bb0 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9668600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f96600cf580 con 0x7f96600cd6c0 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46178/0 (socket says 172.21.15.94:46178) 2024-05-27T12:36:10.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 -- 172.21.15.94:0/2643492660 learned_addr learned my addr 172.21.15.94:0/2643492660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9666600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9660144bb0 0x7f966006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 -- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96600cd6c0 msgr2=0x7f966006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96600cd6c0 0x7f966006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 -- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9660144bb0 msgr2=0x7f966006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9660144bb0 0x7f966006e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.652+0000 7f9665c00640 1 -- 172.21.15.94:0/2643492660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600c8180 con 0x7f9660072630 2024-05-27T12:36:10.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f9665c00640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9654009d90 tx=0x7f9654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:10.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965403f070 con 0x7f9660072630 2024-05-27T12:36:10.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9654031c40 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96601d08a0 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654009450 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.653+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601d0e60 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.654+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9660075460 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.655+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9654044460 con 0x7f9660072630 2024-05-27T12:36:10.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.656+0000 7f965d600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 0x7f964006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:10.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.656+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f96540ad2c0 con 0x7f9660072630 2024-05-27T12:36:10.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.657+0000 7f9665200640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 0x7f964006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:10.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.658+0000 7f9665200640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 0x7f964006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f964c000f60 tx=0x7f964c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:10.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.661+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f965407b780 con 0x7f9660072630 2024-05-27T12:36:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.903+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96601e2cc0 con 0x7f9660072630 2024-05-27T12:36:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.904+0000 7f965d600640 1 -- 172.21.15.94:0/2643492660 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f96540ad0e0 con 0x7f9660072630 2024-05-27T12:36:10.904 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:10.904 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1231254464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1231254464},{"type":"v1","addr":"172.21.15.94:6835","nonce":1231254464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/750013346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":750013346},{"type":"v1","addr":"172.21.15.97:6837","nonce":750013346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/956232445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":956232445},{"type":"v1","addr":"172.21.15.97:6835","nonce":956232445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:36:06.826950+0000","modified":"2024-05-27T12:36:09.894453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.94:6834/2304976623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2304976623},{"type":"v1","addr":"172.21.15.94:6834","nonce":2304976623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4971,"qdb_cluster":[4971]},"id":2}]} 2024-05-27T12:36:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 msgr2=0x7f964006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 0x7f964006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f964c000f60 tx=0x7f964c0079d0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 msgr2=0x7f96600c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9654009d90 tx=0x7f9654004a40 comp rx=0 tx=0).stop 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 shutdown_connections 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f964006c8f0 0x7f964006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9660144bb0 0x7f966006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96600cd6c0 0x7f966006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 --2- 172.21.15.94:0/2643492660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9660072630 0x7f96600c7c40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:10.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 >> 172.21.15.94:0/2643492660 conn(0x7f96600b3830 msgr2=0x7f966013fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:10.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 shutdown_connections 2024-05-27T12:36:10.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:10.907+0000 7f9668600640 1 -- 172.21.15.94:0/2643492660 wait complete. 2024-05-27T12:36:10.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:36:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.124+0000 7f7f23200640 1 Processor -- start 2024-05-27T12:36:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.125+0000 7f7f23200640 1 -- start start 2024-05-27T12:36:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f23200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f23200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7f240d4ae0 0x7f7f240cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f23200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2415ee10 con 0x7f7f240b4510 2024-05-27T12:36:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f23200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240d2830 con 0x7f7f240d4ae0 2024-05-27T12:36:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f23200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240d2a60 con 0x7f7f2415ea30 2024-05-27T12:36:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f22800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7f240d4ae0 0x7f7f240cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f22800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f7f240d4ae0 0x7f7f240cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46194/0 (socket says 172.21.15.94:46194) 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.126+0000 7f7f22800640 1 -- 172.21.15.94:0/99508564 learned_addr learned my addr 172.21.15.94:0/99508564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.127+0000 7f7f21e00640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.127+0000 7f7f22800640 1 -- 172.21.15.94:0/99508564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 msgr2=0x7f7f240d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.127+0000 7f7f22800640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.127+0000 7f7f22800640 1 -- 172.21.15.94:0/99508564 >> v1:172.21.15.94:6789/0 conn(0x7f7f240b4510 legacy=0x7f7f240d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.127+0000 7f7f22800640 1 -- 172.21.15.94:0/99508564 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240d2d40 con 0x7f7f240d4ae0 2024-05-27T12:36:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.128+0000 7f7f22800640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240d4ae0 0x7f7f240cf610 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009870 tx=0x7f7f1c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2eb1f97ede16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.128+0000 7f7f20a00640 1 -- 172.21.15.94:0/99508564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c037070 con 0x7f7f240d4ae0 2024-05-27T12:36:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.128+0000 7f7f20a00640 1 -- 172.21.15.94:0/99508564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c02ae50 con 0x7f7f240d4ae0 2024-05-27T12:36:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.128+0000 7f7f20a00640 1 -- 172.21.15.94:0/99508564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c032860 con 0x7f7f240d4ae0 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 -- 172.21.15.94:0/99508564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240d4ae0 msgr2=0x7f7f240cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240d4ae0 0x7f7f240cf610 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009870 tx=0x7f7f1c02a300 comp rx=0 tx=0).stop 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 -- 172.21.15.94:0/99508564 shutdown_connections 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240d4ae0 0x7f7f240cf610 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 --2- 172.21.15.94:0/99508564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 -- 172.21.15.94:0/99508564 >> 172.21.15.94:0/99508564 conn(0x7f7f2409da70 msgr2=0x7f7f2408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 -- 172.21.15.94:0/99508564 shutdown_connections 2024-05-27T12:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.129+0000 7f7f23200640 1 -- 172.21.15.94:0/99508564 wait complete. 2024-05-27T12:36:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.131+0000 7f7f23200640 1 Processor -- start 2024-05-27T12:36:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.131+0000 7f7f23200640 1 -- start start 2024-05-27T12:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 0x7f7f240b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d35e0 con 0x7f7f240b4510 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d3300 con 0x7f7f240d4ae0 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.132+0000 7f7f23200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240cfb50 con 0x7f7f2415ea30 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46204/0 (socket says 172.21.15.94:46204) 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 -- 172.21.15.94:0/1684397006 learned_addr learned my addr 172.21.15.94:0/1684397006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 -- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 msgr2=0x7f7f240b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f22800640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240b9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 -- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 msgr2=0x7f7f240b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21400640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 0x7f7f240b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 0x7f7f240b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21e00640 1 -- 172.21.15.94:0/1684397006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240b7a90 con 0x7f7f240b4510 2024-05-27T12:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.133+0000 7f7f21400640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 0x7f7f240b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f21e00640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d5f0 tx=0x7f7f0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0125f0 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c019070 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0129d0 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f240b7d70 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.134+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f240b8330 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.135+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24075460 con 0x7f7f240b4510 2024-05-27T12:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.136+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f0c012b70 con 0x7f7f240b4510 2024-05-27T12:36:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.137+0000 7f7f1a000640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 0x7f7f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.137+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7f0c08bf20 con 0x7f7f240b4510 2024-05-27T12:36:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.138+0000 7f7f21400640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 0x7f7f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.139+0000 7f7f21400640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 0x7f7f0006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f240cd030 tx=0x7f7f14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.142+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f0c05a340 con 0x7f7f240b4510 2024-05-27T12:36:11.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.340+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f240b3e20 con 0x7f7f240b4510 2024-05-27T12:36:11.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.342+0000 7f7f1a000640 1 -- 172.21.15.94:0/1684397006 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16621 (secure 0 0 0) 0x7f7f0c059c40 con 0x7f7f240b4510 2024-05-27T12:36:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:11.342 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:36:08.390039+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-27T12:36:04.679045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-27T12:36:05.799460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:36:11.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 msgr2=0x7f7f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 0x7f7f0006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f240cd030 tx=0x7f7f14009040 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 msgr2=0x7f7f240cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d5f0 tx=0x7f7f0c00dab0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 shutdown_connections 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7f0006c8f0 0x7f7f0006edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f240d4ae0 0x7f7f240b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f2415ea30 0x7f7f240b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 --2- 172.21.15.94:0/1684397006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f240b4510 0x7f7f240cc870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 >> 172.21.15.94:0/1684397006 conn(0x7f7f2409da70 msgr2=0x7f7f240d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 shutdown_connections 2024-05-27T12:36:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.345+0000 7f7f23200640 1 -- 172.21.15.94:0/1684397006 wait complete. 2024-05-27T12:36:11.363 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-27T12:36:11.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:36:11.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.567+0000 7f6259800640 1 Processor -- start 2024-05-27T12:36:11.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.567+0000 7f6259800640 1 -- start start 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6259800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f62541460d0 0x7f6254160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6259800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6259800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62540ceae0 con 0x7f6254145cf0 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6259800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f62541602c0 con 0x7f62541460d0 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6259800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f62541605a0 con 0x7f6254071d60 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6253400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f62541460d0 0x7f6254160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6253400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f62541460d0 0x7f6254160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46210/0 (socket says 172.21.15.94:46210) 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6253400640 1 -- 172.21.15.94:0/3456736657 learned_addr learned my addr 172.21.15.94:0/3456736657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.569+0000 7f6253e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6254145cf0 0x7f62540ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36588/0 (socket says 172.21.15.94:36588) 2024-05-27T12:36:11.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.570+0000 7f6253400640 1 -- 172.21.15.94:0/3456736657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 msgr2=0x7f62540721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:11.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.570+0000 7f6253400640 1 --2- 172.21.15.94:0/3456736657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.570+0000 7f6253400640 1 -- 172.21.15.94:0/3456736657 >> v1:172.21.15.94:6789/0 conn(0x7f6254145cf0 legacy=0x7f62540ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.570+0000 7f6253400640 1 -- 172.21.15.94:0/3456736657 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254160880 con 0x7f62541460d0 2024-05-27T12:36:11.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.570+0000 7f6252a00640 1 -- 172.21.15.94:0/3456736657 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272368288 0 0) 0x7f62540ceae0 con 0x7f6254145cf0 2024-05-27T12:36:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.571+0000 7f6253400640 1 --2- 172.21.15.94:0/3456736657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62541460d0 0x7f6254160da0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6240009870 tx=0x7f624002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4208105336de20c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.571+0000 7f6252a00640 1 -- 172.21.15.94:0/3456736657 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240037070 con 0x7f62541460d0 2024-05-27T12:36:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.571+0000 7f6252a00640 1 -- 172.21.15.94:0/3456736657 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240032440 con 0x7f62541460d0 2024-05-27T12:36:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.571+0000 7f6252a00640 1 -- 172.21.15.94:0/3456736657 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240032810 con 0x7f62541460d0 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 -- 172.21.15.94:0/3456736657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62541460d0 msgr2=0x7f6254160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 --2- 172.21.15.94:0/3456736657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62541460d0 0x7f6254160da0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6240009870 tx=0x7f624002a270 comp rx=0 tx=0).stop 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 -- 172.21.15.94:0/3456736657 shutdown_connections 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 --2- 172.21.15.94:0/3456736657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 --2- 172.21.15.94:0/3456736657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62541460d0 0x7f6254160da0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 -- 172.21.15.94:0/3456736657 >> 172.21.15.94:0/3456736657 conn(0x7f62540b2160 msgr2=0x7f62540b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 -- 172.21.15.94:0/3456736657 shutdown_connections 2024-05-27T12:36:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.572+0000 7f6259800640 1 -- 172.21.15.94:0/3456736657 wait complete. 2024-05-27T12:36:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.574+0000 7f6259800640 1 Processor -- start 2024-05-27T12:36:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.574+0000 7f6259800640 1 -- start start 2024-05-27T12:36:11.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62541460d0 0x7f625415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f625406eb00 con 0x7f6254145cf0 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f62540cedc0 con 0x7f62541460d0 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6259800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f625406ec80 con 0x7f6254071d60 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46212/0 (socket says 172.21.15.94:46212) 2024-05-27T12:36:11.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 -- 172.21.15.94:0/2409019727 learned_addr learned my addr 172.21.15.94:0/2409019727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253e00640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 -- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 msgr2=0x7f62540c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 -- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62541460d0 msgr2=0x7f625415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62541460d0 0x7f625415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.575+0000 7f6253400640 1 -- 172.21.15.94:0/2409019727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62541f53a0 con 0x7f6254145cf0 2024-05-27T12:36:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f6253400640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6240009870 tx=0x7f62400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240041070 con 0x7f6254145cf0 2024-05-27T12:36:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624003c870 con 0x7f6254145cf0 2024-05-27T12:36:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240037040 con 0x7f6254145cf0 2024-05-27T12:36:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541f5680 con 0x7f6254145cf0 2024-05-27T12:36:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.576+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541f5c40 con 0x7f6254145cf0 2024-05-27T12:36:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.577+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6254075460 con 0x7f6254145cf0 2024-05-27T12:36:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.578+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6240044070 con 0x7f6254145cf0 2024-05-27T12:36:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.579+0000 7f624be00640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 0x7f623406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.579+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f62400ac7b0 con 0x7f6254145cf0 2024-05-27T12:36:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.580+0000 7f6253e00640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 0x7f623406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.581+0000 7f6253e00640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 0x7f623406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6244007fd0 tx=0x7f6244009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:11.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.584+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f624007ac60 con 0x7f6254145cf0 2024-05-27T12:36:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.783+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6254072e00 con 0x7f623406c920 2024-05-27T12:36:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.804+0000 7f624be00640 1 -- 172.21.15.94:0/2409019727 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f6254072e00 con 0x7f623406c920 2024-05-27T12:36:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:11.808 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:36:11.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.808+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 msgr2=0x7f623406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.808+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 0x7f623406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6244007fd0 tx=0x7f6244009040 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.808+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 msgr2=0x7f625415cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.808+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6240009870 tx=0x7f62400046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 shutdown_connections 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f623406c920 0x7f623406ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62541460d0 0x7f625415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6254145cf0 0x7f625415cb80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 --2- 172.21.15.94:0/2409019727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6254071d60 0x7f62540c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 >> 172.21.15.94:0/2409019727 conn(0x7f62540b2160 msgr2=0x7f62540721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 shutdown_connections 2024-05-27T12:36:11.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:11.809+0000 7f6259800640 1 -- 172.21.15.94:0/2409019727 wait complete. 2024-05-27T12:36:11.826 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-27T12:36:10.883672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3948544,"data_stored":2236530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536186"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380979+0000","last_change":"2024-05-27T12:36:06.380979+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380979+0000","last_undegraded":"2024-05-27T12:36:06.380979+0000","last_fullsized":"2024-05-27T12:36:06.380979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381776+0000","last_change":"2024-05-27T12:36:06.381776+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381776+0000","last_undegraded":"2024-05-27T12:36:06.381776+0000","last_fullsized":"2024-05-27T12:36:06.381776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379465+0000","last_change":"2024-05-27T12:36:06.379465+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379465+0000","last_undegraded":"2024-05-27T12:36:06.379465+0000","last_fullsized":"2024-05-27T12:36:06.379465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374883+0000","last_change":"2024-05-27T12:36:06.374883+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374883+0000","last_undegraded":"2024-05-27T12:36:06.374883+0000","last_fullsized":"2024-05-27T12:36:06.374883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374479+0000","last_change":"2024-05-27T12:36:06.374479+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374479+0000","last_undegraded":"2024-05-27T12:36:06.374479+0000","last_fullsized":"2024-05-27T12:36:06.374479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377579+0000","last_change":"2024-05-27T12:36:06.377579+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377579+0000","last_undegraded":"2024-05-27T12:36:06.377579+0000","last_fullsized":"2024-05-27T12:36:06.377579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.385452+0000","last_change":"2024-05-27T12:36:06.385452+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.385452+0000","last_undegraded":"2024-05-27T12:36:06.385452+0000","last_fullsized":"2024-05-27T12:36:06.385452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387969+0000","last_change":"2024-05-27T12:36:06.387969+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387969+0000","last_undegraded":"2024-05-27T12:36:06.387969+0000","last_fullsized":"2024-05-27T12:36:06.387969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382637+0000","last_change":"2024-05-27T12:36:06.382637+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382637+0000","last_undegraded":"2024-05-27T12:36:06.382637+0000","last_fullsized":"2024-05-27T12:36:06.382637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379941+0000","last_change":"2024-05-27T12:36:06.379941+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379941+0000","last_undegraded":"2024-05-27T12:36:06.379941+0000","last_fullsized":"2024-05-27T12:36:06.379941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.389103+0000","last_change":"2024-05-27T12:36:06.389103+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.389103+0000","last_undegraded":"2024-05-27T12:36:06.389103+0000","last_fullsized":"2024-05-27T12:36:06.389103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381444+0000","last_change":"2024-05-27T12:36:06.381444+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381444+0000","last_undegraded":"2024-05-27T12:36:06.381444+0000","last_fullsized":"2024-05-27T12:36:06.381444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373801+0000","last_change":"2024-05-27T12:36:06.373801+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373801+0000","last_undegraded":"2024-05-27T12:36:06.373801+0000","last_fullsized":"2024-05-27T12:36:06.373801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376906+0000","last_change":"2024-05-27T12:36:06.376906+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376906+0000","last_undegraded":"2024-05-27T12:36:06.376906+0000","last_fullsized":"2024-05-27T12:36:06.376906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377394+0000","last_change":"2024-05-27T12:36:06.377394+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377394+0000","last_undegraded":"2024-05-27T12:36:06.377394+0000","last_fullsized":"2024-05-27T12:36:06.377394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384646+0000","last_change":"2024-05-27T12:36:06.384646+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384646+0000","last_undegraded":"2024-05-27T12:36:06.384646+0000","last_fullsized":"2024-05-27T12:36:06.384646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.391236+0000","last_change":"2024-05-27T12:36:06.391236+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.391236+0000","last_undegraded":"2024-05-27T12:36:06.391236+0000","last_fullsized":"2024-05-27T12:36:06.391236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381233+0000","last_change":"2024-05-27T12:36:06.381233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381233+0000","last_undegraded":"2024-05-27T12:36:06.381233+0000","last_fullsized":"2024-05-27T12:36:06.381233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376290+0000","last_change":"2024-05-27T12:36:06.376290+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376290+0000","last_undegraded":"2024-05-27T12:36:06.376290+0000","last_fullsized":"2024-05-27T12:36:06.376290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376387+0000","last_change":"2024-05-27T12:36:06.376387+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376387+0000","last_undegraded":"2024-05-27T12:36:06.376387+0000","last_fullsized":"2024-05-27T12:36:06.376387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381597+0000","last_change":"2024-05-27T12:36:06.381597+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381597+0000","last_undegraded":"2024-05-27T12:36:06.381597+0000","last_fullsized":"2024-05-27T12:36:06.381597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384281+0000","last_change":"2024-05-27T12:36:06.384281+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384281+0000","last_undegraded":"2024-05-27T12:36:06.384281+0000","last_fullsized":"2024-05-27T12:36:06.384281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373812+0000","last_change":"2024-05-27T12:36:06.373812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373812+0000","last_undegraded":"2024-05-27T12:36:06.373812+0000","last_fullsized":"2024-05-27T12:36:06.373812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376671+0000","last_change":"2024-05-27T12:36:06.376671+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376671+0000","last_undegraded":"2024-05-27T12:36:06.376671+0000","last_fullsized":"2024-05-27T12:36:06.376671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381990+0000","last_change":"2024-05-27T12:36:06.381990+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381990+0000","last_undegraded":"2024-05-27T12:36:06.381990+0000","last_fullsized":"2024-05-27T12:36:06.381990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380336+0000","last_change":"2024-05-27T12:36:06.380336+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380336+0000","last_undegraded":"2024-05-27T12:36:06.380336+0000","last_fullsized":"2024-05-27T12:36:06.380336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382812+0000","last_change":"2024-05-27T12:36:06.382812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382812+0000","last_undegraded":"2024-05-27T12:36:06.382812+0000","last_fullsized":"2024-05-27T12:36:06.382812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380974+0000","last_change":"2024-05-27T12:36:06.380974+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380974+0000","last_undegraded":"2024-05-27T12:36:06.380974+0000","last_fullsized":"2024-05-27T12:36:06.380974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381165+0000","last_change":"2024-05-27T12:36:06.381165+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381165+0000","last_undegraded":"2024-05-27T12:36:06.381165+0000","last_fullsized":"2024-05-27T12:36:06.381165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384415+0000","last_change":"2024-05-27T12:36:06.384415+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384415+0000","last_undegraded":"2024-05-27T12:36:06.384415+0000","last_fullsized":"2024-05-27T12:36:06.384415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378202+0000","last_change":"2024-05-27T12:36:06.378202+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378202+0000","last_undegraded":"2024-05-27T12:36:06.378202+0000","last_fullsized":"2024-05-27T12:36:06.378202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387285+0000","last_change":"2024-05-27T12:36:06.387285+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387285+0000","last_undegraded":"2024-05-27T12:36:06.387285+0000","last_fullsized":"2024-05-27T12:36:06.387285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373007+0000","last_change":"2024-05-27T12:36:06.373007+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373007+0000","last_undegraded":"2024-05-27T12:36:06.373007+0000","last_fullsized":"2024-05-27T12:36:06.373007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402763+0000","last_change":"2024-05-27T12:36:06.399603+0000","last_active":"2024-05-27T12:36:06.402763+0000","last_peered":"2024-05-27T12:36:06.402763+0000","last_clean":"2024-05-27T12:36:06.402763+0000","last_became_active":"2024-05-27T12:36:06.399286+0000","last_became_peered":"2024-05-27T12:36:06.399286+0000","last_unstale":"2024-05-27T12:36:06.402763+0000","last_undegraded":"2024-05-27T12:36:06.402763+0000","last_fullsized":"2024-05-27T12:36:06.402763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377354+0000","last_change":"2024-05-27T12:36:06.377354+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377354+0000","last_undegraded":"2024-05-27T12:36:06.377354+0000","last_fullsized":"2024-05-27T12:36:06.377354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399712+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.399712+0000","last_peered":"2024-05-27T12:36:06.399712+0000","last_clean":"2024-05-27T12:36:06.399712+0000","last_became_active":"2024-05-27T12:36:06.398355+0000","last_became_peered":"2024-05-27T12:36:06.398355+0000","last_unstale":"2024-05-27T12:36:06.399712+0000","last_undegraded":"2024-05-27T12:36:06.399712+0000","last_fullsized":"2024-05-27T12:36:06.399712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373152+0000","last_change":"2024-05-27T12:36:06.373152+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373152+0000","last_undegraded":"2024-05-27T12:36:06.373152+0000","last_fullsized":"2024-05-27T12:36:06.373152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395682+0000","last_change":"2024-05-27T12:36:06.393410+0000","last_active":"2024-05-27T12:36:06.395682+0000","last_peered":"2024-05-27T12:36:06.395682+0000","last_clean":"2024-05-27T12:36:06.395682+0000","last_became_active":"2024-05-27T12:36:06.392937+0000","last_became_peered":"2024-05-27T12:36:06.392937+0000","last_unstale":"2024-05-27T12:36:06.395682+0000","last_undegraded":"2024-05-27T12:36:06.395682+0000","last_fullsized":"2024-05-27T12:36:06.395682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382381+0000","last_change":"2024-05-27T12:36:06.382381+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382381+0000","last_undegraded":"2024-05-27T12:36:06.382381+0000","last_fullsized":"2024-05-27T12:36:06.382381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.398830+0000","last_change":"2024-05-27T12:36:06.395813+0000","last_active":"2024-05-27T12:36:06.398830+0000","last_peered":"2024-05-27T12:36:06.398830+0000","last_clean":"2024-05-27T12:36:06.398830+0000","last_became_active":"2024-05-27T12:36:06.392355+0000","last_became_peered":"2024-05-27T12:36:06.392355+0000","last_unstale":"2024-05-27T12:36:06.398830+0000","last_undegraded":"2024-05-27T12:36:06.398830+0000","last_fullsized":"2024-05-27T12:36:06.398830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383806+0000","last_change":"2024-05-27T12:36:06.383806+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383806+0000","last_undegraded":"2024-05-27T12:36:06.383806+0000","last_fullsized":"2024-05-27T12:36:06.383806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402620+0000","last_change":"2024-05-27T12:36:06.394640+0000","last_active":"2024-05-27T12:36:06.402620+0000","last_peered":"2024-05-27T12:36:06.402620+0000","last_clean":"2024-05-27T12:36:06.402620+0000","last_became_active":"2024-05-27T12:36:06.394297+0000","last_became_peered":"2024-05-27T12:36:06.394297+0000","last_unstale":"2024-05-27T12:36:06.402620+0000","last_undegraded":"2024-05-27T12:36:06.402620+0000","last_fullsized":"2024-05-27T12:36:06.402620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:17.713278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382750+0000","last_change":"2024-05-27T12:36:06.382750+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382750+0000","last_undegraded":"2024-05-27T12:36:06.382750+0000","last_fullsized":"2024-05-27T12:36:06.382750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402477+0000","last_change":"2024-05-27T12:36:06.394969+0000","last_active":"2024-05-27T12:36:06.402477+0000","last_peered":"2024-05-27T12:36:06.402477+0000","last_clean":"2024-05-27T12:36:06.402477+0000","last_became_active":"2024-05-27T12:36:06.394600+0000","last_became_peered":"2024-05-27T12:36:06.394600+0000","last_unstale":"2024-05-27T12:36:06.402477+0000","last_undegraded":"2024-05-27T12:36:06.402477+0000","last_fullsized":"2024-05-27T12:36:06.402477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:48.343915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376119+0000","last_change":"2024-05-27T12:36:06.376119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376119+0000","last_undegraded":"2024-05-27T12:36:06.376119+0000","last_fullsized":"2024-05-27T12:36:06.376119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.385609+0000","last_change":"2024-05-27T12:36:06.383317+0000","last_active":"2024-05-27T12:36:06.385609+0000","last_peered":"2024-05-27T12:36:06.385609+0000","last_clean":"2024-05-27T12:36:06.385609+0000","last_became_active":"2024-05-27T12:36:06.383006+0000","last_became_peered":"2024-05-27T12:36:06.383006+0000","last_unstale":"2024-05-27T12:36:06.385609+0000","last_undegraded":"2024-05-27T12:36:06.385609+0000","last_fullsized":"2024-05-27T12:36:06.385609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374894+0000","last_change":"2024-05-27T12:36:06.374894+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374894+0000","last_undegraded":"2024-05-27T12:36:06.374894+0000","last_fullsized":"2024-05-27T12:36:06.374894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402610+0000","last_change":"2024-05-27T12:36:06.398382+0000","last_active":"2024-05-27T12:36:06.402610+0000","last_peered":"2024-05-27T12:36:06.402610+0000","last_clean":"2024-05-27T12:36:06.402610+0000","last_became_active":"2024-05-27T12:36:06.396324+0000","last_became_peered":"2024-05-27T12:36:06.396324+0000","last_unstale":"2024-05-27T12:36:06.402610+0000","last_undegraded":"2024-05-27T12:36:06.402610+0000","last_fullsized":"2024-05-27T12:36:06.402610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:50:46.342036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384618+0000","last_change":"2024-05-27T12:36:06.384618+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384618+0000","last_undegraded":"2024-05-27T12:36:06.384618+0000","last_fullsized":"2024-05-27T12:36:06.384618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401315+0000","last_change":"2024-05-27T12:36:06.400135+0000","last_active":"2024-05-27T12:36:06.401315+0000","last_peered":"2024-05-27T12:36:06.401315+0000","last_clean":"2024-05-27T12:36:06.401315+0000","last_became_active":"2024-05-27T12:36:06.399910+0000","last_became_peered":"2024-05-27T12:36:06.399910+0000","last_unstale":"2024-05-27T12:36:06.401315+0000","last_undegraded":"2024-05-27T12:36:06.401315+0000","last_fullsized":"2024-05-27T12:36:06.401315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:45:46.769699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376474+0000","last_change":"2024-05-27T12:36:06.376474+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376474+0000","last_undegraded":"2024-05-27T12:36:06.376474+0000","last_fullsized":"2024-05-27T12:36:06.376474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.386818+0000","last_change":"2024-05-27T12:36:06.383845+0000","last_active":"2024-05-27T12:36:06.386818+0000","last_peered":"2024-05-27T12:36:06.386818+0000","last_clean":"2024-05-27T12:36:06.386818+0000","last_became_active":"2024-05-27T12:36:06.383394+0000","last_became_peered":"2024-05-27T12:36:06.383394+0000","last_unstale":"2024-05-27T12:36:06.386818+0000","last_undegraded":"2024-05-27T12:36:06.386818+0000","last_fullsized":"2024-05-27T12:36:06.386818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373233+0000","last_change":"2024-05-27T12:36:06.373233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373233+0000","last_undegraded":"2024-05-27T12:36:06.373233+0000","last_fullsized":"2024-05-27T12:36:06.373233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402769+0000","last_change":"2024-05-27T12:36:06.398488+0000","last_active":"2024-05-27T12:36:06.402769+0000","last_peered":"2024-05-27T12:36:06.402769+0000","last_clean":"2024-05-27T12:36:06.402769+0000","last_became_active":"2024-05-27T12:36:06.397511+0000","last_became_peered":"2024-05-27T12:36:06.397511+0000","last_unstale":"2024-05-27T12:36:06.402769+0000","last_undegraded":"2024-05-27T12:36:06.402769+0000","last_fullsized":"2024-05-27T12:36:06.402769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:40.771809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379981+0000","last_change":"2024-05-27T12:36:06.379981+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379981+0000","last_undegraded":"2024-05-27T12:36:06.379981+0000","last_fullsized":"2024-05-27T12:36:06.379981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397658+0000","last_change":"2024-05-27T12:36:06.393888+0000","last_active":"2024-05-27T12:36:06.397658+0000","last_peered":"2024-05-27T12:36:06.397658+0000","last_clean":"2024-05-27T12:36:06.397658+0000","last_became_active":"2024-05-27T12:36:06.393567+0000","last_became_peered":"2024-05-27T12:36:06.393567+0000","last_unstale":"2024-05-27T12:36:06.397658+0000","last_undegraded":"2024-05-27T12:36:06.397658+0000","last_fullsized":"2024-05-27T12:36:06.397658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374467+0000","last_change":"2024-05-27T12:36:06.374467+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374467+0000","last_undegraded":"2024-05-27T12:36:06.374467+0000","last_fullsized":"2024-05-27T12:36:06.374467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402643+0000","last_change":"2024-05-27T12:36:06.396297+0000","last_active":"2024-05-27T12:36:06.402643+0000","last_peered":"2024-05-27T12:36:06.402643+0000","last_clean":"2024-05-27T12:36:06.402643+0000","last_became_active":"2024-05-27T12:36:06.395605+0000","last_became_peered":"2024-05-27T12:36:06.395605+0000","last_unstale":"2024-05-27T12:36:06.402643+0000","last_undegraded":"2024-05-27T12:36:06.402643+0000","last_fullsized":"2024-05-27T12:36:06.402643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:04.614416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373670+0000","last_change":"2024-05-27T12:36:06.373670+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373670+0000","last_undegraded":"2024-05-27T12:36:06.373670+0000","last_fullsized":"2024-05-27T12:36:06.373670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396083+0000","last_change":"2024-05-27T12:36:06.391699+0000","last_active":"2024-05-27T12:36:06.396083+0000","last_peered":"2024-05-27T12:36:06.396083+0000","last_clean":"2024-05-27T12:36:06.396083+0000","last_became_active":"2024-05-27T12:36:06.391285+0000","last_became_peered":"2024-05-27T12:36:06.391285+0000","last_unstale":"2024-05-27T12:36:06.396083+0000","last_undegraded":"2024-05-27T12:36:06.396083+0000","last_fullsized":"2024-05-27T12:36:06.396083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386703+0000","last_change":"2024-05-27T12:36:06.386703+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386703+0000","last_undegraded":"2024-05-27T12:36:06.386703+0000","last_fullsized":"2024-05-27T12:36:06.386703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400607+0000","last_change":"2024-05-27T12:36:06.399046+0000","last_active":"2024-05-27T12:36:06.400607+0000","last_peered":"2024-05-27T12:36:06.400607+0000","last_clean":"2024-05-27T12:36:06.400607+0000","last_became_active":"2024-05-27T12:36:06.396837+0000","last_became_peered":"2024-05-27T12:36:06.396837+0000","last_unstale":"2024-05-27T12:36:06.400607+0000","last_undegraded":"2024-05-27T12:36:06.400607+0000","last_fullsized":"2024-05-27T12:36:06.400607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376569+0000","last_change":"2024-05-27T12:36:06.376569+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376569+0000","last_undegraded":"2024-05-27T12:36:06.376569+0000","last_fullsized":"2024-05-27T12:36:06.376569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400216+0000","last_change":"2024-05-27T12:36:06.397002+0000","last_active":"2024-05-27T12:36:06.400216+0000","last_peered":"2024-05-27T12:36:06.400216+0000","last_clean":"2024-05-27T12:36:06.400216+0000","last_became_active":"2024-05-27T12:36:06.396513+0000","last_became_peered":"2024-05-27T12:36:06.396513+0000","last_unstale":"2024-05-27T12:36:06.400216+0000","last_undegraded":"2024-05-27T12:36:06.400216+0000","last_fullsized":"2024-05-27T12:36:06.400216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382122+0000","last_change":"2024-05-27T12:36:06.382122+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382122+0000","last_undegraded":"2024-05-27T12:36:06.382122+0000","last_fullsized":"2024-05-27T12:36:06.382122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395817+0000","last_change":"2024-05-27T12:36:06.390945+0000","last_active":"2024-05-27T12:36:06.395817+0000","last_peered":"2024-05-27T12:36:06.395817+0000","last_clean":"2024-05-27T12:36:06.395817+0000","last_became_active":"2024-05-27T12:36:06.390575+0000","last_became_peered":"2024-05-27T12:36:06.390575+0000","last_unstale":"2024-05-27T12:36:06.395817+0000","last_undegraded":"2024-05-27T12:36:06.395817+0000","last_fullsized":"2024-05-27T12:36:06.395817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384033+0000","last_change":"2024-05-27T12:36:06.384033+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384033+0000","last_undegraded":"2024-05-27T12:36:06.384033+0000","last_fullsized":"2024-05-27T12:36:06.384033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402551+0000","last_change":"2024-05-27T12:36:06.396687+0000","last_active":"2024-05-27T12:36:06.402551+0000","last_peered":"2024-05-27T12:36:06.402551+0000","last_clean":"2024-05-27T12:36:06.402551+0000","last_became_active":"2024-05-27T12:36:06.395340+0000","last_became_peered":"2024-05-27T12:36:06.395340+0000","last_unstale":"2024-05-27T12:36:06.402551+0000","last_undegraded":"2024-05-27T12:36:06.402551+0000","last_fullsized":"2024-05-27T12:36:06.402551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:14.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373578+0000","last_change":"2024-05-27T12:36:06.373578+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373578+0000","last_undegraded":"2024-05-27T12:36:06.373578+0000","last_fullsized":"2024-05-27T12:36:06.373578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399923+0000","last_change":"2024-05-27T12:36:06.393791+0000","last_active":"2024-05-27T12:36:06.399923+0000","last_peered":"2024-05-27T12:36:06.399923+0000","last_clean":"2024-05-27T12:36:06.399923+0000","last_became_active":"2024-05-27T12:36:06.391768+0000","last_became_peered":"2024-05-27T12:36:06.391768+0000","last_unstale":"2024-05-27T12:36:06.399923+0000","last_undegraded":"2024-05-27T12:36:06.399923+0000","last_fullsized":"2024-05-27T12:36:06.399923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:11:10.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380582+0000","last_change":"2024-05-27T12:36:06.380582+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380582+0000","last_undegraded":"2024-05-27T12:36:06.380582+0000","last_fullsized":"2024-05-27T12:36:06.380582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396504+0000","last_change":"2024-05-27T12:36:06.388667+0000","last_active":"2024-05-27T12:36:06.396504+0000","last_peered":"2024-05-27T12:36:06.396504+0000","last_clean":"2024-05-27T12:36:06.396504+0000","last_became_active":"2024-05-27T12:36:06.388369+0000","last_became_peered":"2024-05-27T12:36:06.388369+0000","last_unstale":"2024-05-27T12:36:06.396504+0000","last_undegraded":"2024-05-27T12:36:06.396504+0000","last_fullsized":"2024-05-27T12:36:06.396504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:49.907565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374535+0000","last_change":"2024-05-27T12:36:06.374535+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374535+0000","last_undegraded":"2024-05-27T12:36:06.374535+0000","last_fullsized":"2024-05-27T12:36:06.374535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394967+0000","last_change":"2024-05-27T12:36:06.391794+0000","last_active":"2024-05-27T12:36:06.394967+0000","last_peered":"2024-05-27T12:36:06.394967+0000","last_clean":"2024-05-27T12:36:06.394967+0000","last_became_active":"2024-05-27T12:36:06.389977+0000","last_became_peered":"2024-05-27T12:36:06.389977+0000","last_unstale":"2024-05-27T12:36:06.394967+0000","last_undegraded":"2024-05-27T12:36:06.394967+0000","last_fullsized":"2024-05-27T12:36:06.394967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373084+0000","last_change":"2024-05-27T12:36:06.373084+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373084+0000","last_undegraded":"2024-05-27T12:36:06.373084+0000","last_fullsized":"2024-05-27T12:36:06.373084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397674+0000","last_change":"2024-05-27T12:36:06.393310+0000","last_active":"2024-05-27T12:36:06.397674+0000","last_peered":"2024-05-27T12:36:06.397674+0000","last_clean":"2024-05-27T12:36:06.397674+0000","last_became_active":"2024-05-27T12:36:06.392014+0000","last_became_peered":"2024-05-27T12:36:06.392014+0000","last_unstale":"2024-05-27T12:36:06.397674+0000","last_undegraded":"2024-05-27T12:36:06.397674+0000","last_fullsized":"2024-05-27T12:36:06.397674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374186+0000","last_change":"2024-05-27T12:36:06.374186+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374186+0000","last_undegraded":"2024-05-27T12:36:06.374186+0000","last_fullsized":"2024-05-27T12:36:06.374186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399777+0000","last_change":"2024-05-27T12:36:06.393400+0000","last_active":"2024-05-27T12:36:06.399777+0000","last_peered":"2024-05-27T12:36:06.399777+0000","last_clean":"2024-05-27T12:36:06.399777+0000","last_became_active":"2024-05-27T12:36:06.392996+0000","last_became_peered":"2024-05-27T12:36:06.392996+0000","last_unstale":"2024-05-27T12:36:06.399777+0000","last_undegraded":"2024-05-27T12:36:06.399777+0000","last_fullsized":"2024-05-27T12:36:06.399777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386244+0000","last_change":"2024-05-27T12:36:06.386244+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386244+0000","last_undegraded":"2024-05-27T12:36:06.386244+0000","last_fullsized":"2024-05-27T12:36:06.386244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400560+0000","last_change":"2024-05-27T12:36:06.396022+0000","last_active":"2024-05-27T12:36:06.400560+0000","last_peered":"2024-05-27T12:36:06.400560+0000","last_clean":"2024-05-27T12:36:06.400560+0000","last_became_active":"2024-05-27T12:36:06.395727+0000","last_became_peered":"2024-05-27T12:36:06.395727+0000","last_unstale":"2024-05-27T12:36:06.400560+0000","last_undegraded":"2024-05-27T12:36:06.400560+0000","last_fullsized":"2024-05-27T12:36:06.400560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:03.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379447+0000","last_change":"2024-05-27T12:36:06.379447+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379447+0000","last_undegraded":"2024-05-27T12:36:06.379447+0000","last_fullsized":"2024-05-27T12:36:06.379447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395838+0000","last_change":"2024-05-27T12:36:06.393032+0000","last_active":"2024-05-27T12:36:06.395838+0000","last_peered":"2024-05-27T12:36:06.395838+0000","last_clean":"2024-05-27T12:36:06.395838+0000","last_became_active":"2024-05-27T12:36:06.392754+0000","last_became_peered":"2024-05-27T12:36:06.392754+0000","last_unstale":"2024-05-27T12:36:06.395838+0000","last_undegraded":"2024-05-27T12:36:06.395838+0000","last_fullsized":"2024-05-27T12:36:06.395838+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379855+0000","last_change":"2024-05-27T12:36:06.379855+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379855+0000","last_undegraded":"2024-05-27T12:36:06.379855+0000","last_fullsized":"2024-05-27T12:36:06.379855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401022+0000","last_change":"2024-05-27T12:36:06.399546+0000","last_active":"2024-05-27T12:36:06.401022+0000","last_peered":"2024-05-27T12:36:06.401022+0000","last_clean":"2024-05-27T12:36:06.401022+0000","last_became_active":"2024-05-27T12:36:06.399228+0000","last_became_peered":"2024-05-27T12:36:06.399228+0000","last_unstale":"2024-05-27T12:36:06.401022+0000","last_undegraded":"2024-05-27T12:36:06.401022+0000","last_fullsized":"2024-05-27T12:36:06.401022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375413+0000","last_change":"2024-05-27T12:36:06.375413+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375413+0000","last_undegraded":"2024-05-27T12:36:06.375413+0000","last_fullsized":"2024-05-27T12:36:06.375413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395586+0000","last_change":"2024-05-27T12:36:06.391239+0000","last_active":"2024-05-27T12:36:06.395586+0000","last_peered":"2024-05-27T12:36:06.395586+0000","last_clean":"2024-05-27T12:36:06.395586+0000","last_became_active":"2024-05-27T12:36:06.390800+0000","last_became_peered":"2024-05-27T12:36:06.390800+0000","last_unstale":"2024-05-27T12:36:06.395586+0000","last_undegraded":"2024-05-27T12:36:06.395586+0000","last_fullsized":"2024-05-27T12:36:06.395586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375292+0000","last_change":"2024-05-27T12:36:06.375292+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375292+0000","last_undegraded":"2024-05-27T12:36:06.375292+0000","last_fullsized":"2024-05-27T12:36:06.375292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.391355+0000","last_change":"2024-05-27T12:36:06.389001+0000","last_active":"2024-05-27T12:36:06.391355+0000","last_peered":"2024-05-27T12:36:06.391355+0000","last_clean":"2024-05-27T12:36:06.391355+0000","last_became_active":"2024-05-27T12:36:06.388700+0000","last_became_peered":"2024-05-27T12:36:06.388700+0000","last_unstale":"2024-05-27T12:36:06.391355+0000","last_undegraded":"2024-05-27T12:36:06.391355+0000","last_fullsized":"2024-05-27T12:36:06.391355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.372335+0000","last_change":"2024-05-27T12:36:06.372335+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.372335+0000","last_undegraded":"2024-05-27T12:36:06.372335+0000","last_fullsized":"2024-05-27T12:36:06.372335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394265+0000","last_change":"2024-05-27T12:36:06.390271+0000","last_active":"2024-05-27T12:36:06.394265+0000","last_peered":"2024-05-27T12:36:06.394265+0000","last_clean":"2024-05-27T12:36:06.394265+0000","last_became_active":"2024-05-27T12:36:06.389933+0000","last_became_peered":"2024-05-27T12:36:06.389933+0000","last_unstale":"2024-05-27T12:36:06.394265+0000","last_undegraded":"2024-05-27T12:36:06.394265+0000","last_fullsized":"2024-05-27T12:36:06.394265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378119+0000","last_change":"2024-05-27T12:36:06.378119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378119+0000","last_undegraded":"2024-05-27T12:36:06.378119+0000","last_fullsized":"2024-05-27T12:36:06.378119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396869+0000","last_change":"2024-05-27T12:36:06.389847+0000","last_active":"2024-05-27T12:36:06.396869+0000","last_peered":"2024-05-27T12:36:06.396869+0000","last_clean":"2024-05-27T12:36:06.396869+0000","last_became_active":"2024-05-27T12:36:06.389431+0000","last_became_peered":"2024-05-27T12:36:06.389431+0000","last_unstale":"2024-05-27T12:36:06.396869+0000","last_undegraded":"2024-05-27T12:36:06.396869+0000","last_fullsized":"2024-05-27T12:36:06.396869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:54:21.911265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373815+0000","last_change":"2024-05-27T12:36:06.373815+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373815+0000","last_undegraded":"2024-05-27T12:36:06.373815+0000","last_fullsized":"2024-05-27T12:36:06.373815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400629+0000","last_change":"2024-05-27T12:36:06.399243+0000","last_active":"2024-05-27T12:36:06.400629+0000","last_peered":"2024-05-27T12:36:06.400629+0000","last_clean":"2024-05-27T12:36:06.400629+0000","last_became_active":"2024-05-27T12:36:06.397252+0000","last_became_peered":"2024-05-27T12:36:06.397252+0000","last_unstale":"2024-05-27T12:36:06.400629+0000","last_undegraded":"2024-05-27T12:36:06.400629+0000","last_fullsized":"2024-05-27T12:36:06.400629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:38.949294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383963+0000","last_change":"2024-05-27T12:36:06.383963+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383963+0000","last_undegraded":"2024-05-27T12:36:06.383963+0000","last_fullsized":"2024-05-27T12:36:06.383963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400637+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.400637+0000","last_peered":"2024-05-27T12:36:06.400637+0000","last_clean":"2024-05-27T12:36:06.400637+0000","last_became_active":"2024-05-27T12:36:06.398205+0000","last_became_peered":"2024-05-27T12:36:06.398205+0000","last_unstale":"2024-05-27T12:36:06.400637+0000","last_undegraded":"2024-05-27T12:36:06.400637+0000","last_fullsized":"2024-05-27T12:36:06.400637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.393078+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:36:06.393078+0000","last_peered":"2024-05-27T12:36:06.393078+0000","last_clean":"2024-05-27T12:36:06.393078+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:36:06.393078+0000","last_undegraded":"2024-05-27T12:36:06.393078+0000","last_fullsized":"2024-05-27T12:36:06.393078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:36:11.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:36:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.069+0000 7f8b30400640 1 Processor -- start 2024-05-27T12:36:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.069+0000 7f8b30400640 1 -- start start 2024-05-27T12:36:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.071+0000 7f8b30400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b2815e920 0x7f8b28162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.071+0000 7f8b30400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b28072630 0x7f8b280ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.071+0000 7f8b30400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2815ed00 con 0x7f8b280b2f10 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.071+0000 7f8b30400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f8b280cd580 con 0x7f8b28072630 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.071+0000 7f8b30400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8b280cd860 con 0x7f8b2815e920 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.072+0000 7f8b2da00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b2815e920 0x7f8b28162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.072+0000 7f8b2e400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b28072630 0x7f8b280ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.072+0000 7f8b2e400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b28072630 0x7f8b280ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:40216/0 (socket says 172.21.15.94:40216) 2024-05-27T12:36:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.072+0000 7f8b2e400640 1 -- 172.21.15.94:0/1608248304 learned_addr learned my addr 172.21.15.94:0/1608248304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.073+0000 7f8b2e400640 1 -- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b2815e920 msgr2=0x7f8b28162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.073+0000 7f8b2e400640 1 --2- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b2815e920 0x7f8b28162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.073+0000 7f8b2e400640 1 -- 172.21.15.94:0/1608248304 >> v1:172.21.15.97:6789/0 conn(0x7f8b280b2f10 legacy=0x7f8b28071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.073+0000 7f8b2e400640 1 -- 172.21.15.94:0/1608248304 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b280cdb40 con 0x7f8b28072630 2024-05-27T12:36:12.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.074+0000 7f8b2e400640 1 --2- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28072630 0x7f8b280ce920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009870 tx=0x7f8b2002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f3a772511f5e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.074+0000 7f8b27e00640 1 -- 172.21.15.94:0/1608248304 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20037070 con 0x7f8b28072630 2024-05-27T12:36:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.074+0000 7f8b27e00640 1 -- 172.21.15.94:0/1608248304 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b20032440 con 0x7f8b28072630 2024-05-27T12:36:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.074+0000 7f8b27e00640 1 -- 172.21.15.94:0/1608248304 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20032810 con 0x7f8b28072630 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 -- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28072630 msgr2=0x7f8b280ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 --2- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28072630 0x7f8b280ce920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009870 tx=0x7f8b2002a250 comp rx=0 tx=0).stop 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 -- 172.21.15.94:0/1608248304 shutdown_connections 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 --2- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b28072630 0x7f8b280ce920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 --2- 172.21.15.94:0/1608248304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b2815e920 0x7f8b28162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 -- 172.21.15.94:0/1608248304 >> 172.21.15.94:0/1608248304 conn(0x7f8b280b3830 msgr2=0x7f8b280b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 -- 172.21.15.94:0/1608248304 shutdown_connections 2024-05-27T12:36:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.075+0000 7f8b30400640 1 -- 172.21.15.94:0/1608248304 wait complete. 2024-05-27T12:36:12.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.077+0000 7f8b30400640 1 Processor -- start 2024-05-27T12:36:12.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.077+0000 7f8b30400640 1 -- start start 2024-05-27T12:36:12.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b28072630 0x7f8b281f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 0x7f8b2814ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28163380 con 0x7f8b2815e920 2024-05-27T12:36:12.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b280ce100 con 0x7f8b280b2f10 2024-05-27T12:36:12.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.078+0000 7f8b30400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8b280cee60 con 0x7f8b28072630 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46224/0 (socket says 172.21.15.94:46224) 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 -- 172.21.15.94:0/3488154451 learned_addr learned my addr 172.21.15.94:0/3488154451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2d000640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 0x7f8b2814ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2da00640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b28072630 0x7f8b281f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 -- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b28072630 msgr2=0x7f8b281f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b28072630 0x7f8b281f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 -- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 msgr2=0x7f8b2814ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 0x7f8b2814ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2e400640 1 -- 172.21.15.94:0/3488154451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b281f5640 con 0x7f8b2815e920 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.079+0000 7f8b2d000640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 0x7f8b2814ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:36:12.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b2e400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009870 tx=0x7f8b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20041020 con 0x7f8b2815e920 2024-05-27T12:36:12.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2003d640 con 0x7f8b2815e920 2024-05-27T12:36:12.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b28152be0 con 0x7f8b2815e920 2024-05-27T12:36:12.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b20037040 con 0x7f8b2815e920 2024-05-27T12:36:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.080+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2814b290 con 0x7f8b2815e920 2024-05-27T12:36:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.081+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b280708c0 con 0x7f8b2815e920 2024-05-27T12:36:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.082+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b20044020 con 0x7f8b2815e920 2024-05-27T12:36:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.083+0000 7f8b25600640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.083+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8b200ac370 con 0x7f8b2815e920 2024-05-27T12:36:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.084+0000 7f8b2da00640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.085+0000 7f8b2da00640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 0x7f8b0806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b28162480 tx=0x7f8b1c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.088+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b280708c0 con 0x7f8b2815e920 2024-05-27T12:36:12.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.281+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b28075460 con 0x7f8b0806c8f0 2024-05-27T12:36:12.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.299+0000 7f8b25600640 1 -- 172.21.15.94:0/3488154451 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f8b28075460 con 0x7f8b0806c8f0 2024-05-27T12:36:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 msgr2=0x7f8b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 0x7f8b0806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b28162480 tx=0x7f8b1c00b040 comp rx=0 tx=0).stop 2024-05-27T12:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 msgr2=0x7f8b2815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009870 tx=0x7f8b200046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 shutdown_connections 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8b0806c8f0 0x7f8b0806edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b280b2f10 0x7f8b2814ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815e920 0x7f8b2815d090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 --2- 172.21.15.94:0/3488154451 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b28072630 0x7f8b281f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 >> 172.21.15.94:0/3488154451 conn(0x7f8b280b3830 msgr2=0x7f8b280cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 shutdown_connections 2024-05-27T12:36:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.306+0000 7f8b30400640 1 -- 172.21.15.94:0/3488154451 wait complete. 2024-05-27T12:36:12.325 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-27T12:36:10.883672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3948544,"data_stored":2236530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536186"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380979+0000","last_change":"2024-05-27T12:36:06.380979+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380979+0000","last_undegraded":"2024-05-27T12:36:06.380979+0000","last_fullsized":"2024-05-27T12:36:06.380979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381776+0000","last_change":"2024-05-27T12:36:06.381776+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381776+0000","last_undegraded":"2024-05-27T12:36:06.381776+0000","last_fullsized":"2024-05-27T12:36:06.381776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379465+0000","last_change":"2024-05-27T12:36:06.379465+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379465+0000","last_undegraded":"2024-05-27T12:36:06.379465+0000","last_fullsized":"2024-05-27T12:36:06.379465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374883+0000","last_change":"2024-05-27T12:36:06.374883+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374883+0000","last_undegraded":"2024-05-27T12:36:06.374883+0000","last_fullsized":"2024-05-27T12:36:06.374883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374479+0000","last_change":"2024-05-27T12:36:06.374479+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374479+0000","last_undegraded":"2024-05-27T12:36:06.374479+0000","last_fullsized":"2024-05-27T12:36:06.374479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377579+0000","last_change":"2024-05-27T12:36:06.377579+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377579+0000","last_undegraded":"2024-05-27T12:36:06.377579+0000","last_fullsized":"2024-05-27T12:36:06.377579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.385452+0000","last_change":"2024-05-27T12:36:06.385452+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.385452+0000","last_undegraded":"2024-05-27T12:36:06.385452+0000","last_fullsized":"2024-05-27T12:36:06.385452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387969+0000","last_change":"2024-05-27T12:36:06.387969+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387969+0000","last_undegraded":"2024-05-27T12:36:06.387969+0000","last_fullsized":"2024-05-27T12:36:06.387969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382637+0000","last_change":"2024-05-27T12:36:06.382637+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382637+0000","last_undegraded":"2024-05-27T12:36:06.382637+0000","last_fullsized":"2024-05-27T12:36:06.382637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379941+0000","last_change":"2024-05-27T12:36:06.379941+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379941+0000","last_undegraded":"2024-05-27T12:36:06.379941+0000","last_fullsized":"2024-05-27T12:36:06.379941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.389103+0000","last_change":"2024-05-27T12:36:06.389103+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.389103+0000","last_undegraded":"2024-05-27T12:36:06.389103+0000","last_fullsized":"2024-05-27T12:36:06.389103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381444+0000","last_change":"2024-05-27T12:36:06.381444+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381444+0000","last_undegraded":"2024-05-27T12:36:06.381444+0000","last_fullsized":"2024-05-27T12:36:06.381444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373801+0000","last_change":"2024-05-27T12:36:06.373801+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373801+0000","last_undegraded":"2024-05-27T12:36:06.373801+0000","last_fullsized":"2024-05-27T12:36:06.373801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376906+0000","last_change":"2024-05-27T12:36:06.376906+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376906+0000","last_undegraded":"2024-05-27T12:36:06.376906+0000","last_fullsized":"2024-05-27T12:36:06.376906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377394+0000","last_change":"2024-05-27T12:36:06.377394+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377394+0000","last_undegraded":"2024-05-27T12:36:06.377394+0000","last_fullsized":"2024-05-27T12:36:06.377394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384646+0000","last_change":"2024-05-27T12:36:06.384646+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384646+0000","last_undegraded":"2024-05-27T12:36:06.384646+0000","last_fullsized":"2024-05-27T12:36:06.384646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.391236+0000","last_change":"2024-05-27T12:36:06.391236+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.391236+0000","last_undegraded":"2024-05-27T12:36:06.391236+0000","last_fullsized":"2024-05-27T12:36:06.391236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381233+0000","last_change":"2024-05-27T12:36:06.381233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381233+0000","last_undegraded":"2024-05-27T12:36:06.381233+0000","last_fullsized":"2024-05-27T12:36:06.381233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376290+0000","last_change":"2024-05-27T12:36:06.376290+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376290+0000","last_undegraded":"2024-05-27T12:36:06.376290+0000","last_fullsized":"2024-05-27T12:36:06.376290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376387+0000","last_change":"2024-05-27T12:36:06.376387+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376387+0000","last_undegraded":"2024-05-27T12:36:06.376387+0000","last_fullsized":"2024-05-27T12:36:06.376387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381597+0000","last_change":"2024-05-27T12:36:06.381597+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381597+0000","last_undegraded":"2024-05-27T12:36:06.381597+0000","last_fullsized":"2024-05-27T12:36:06.381597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384281+0000","last_change":"2024-05-27T12:36:06.384281+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384281+0000","last_undegraded":"2024-05-27T12:36:06.384281+0000","last_fullsized":"2024-05-27T12:36:06.384281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373812+0000","last_change":"2024-05-27T12:36:06.373812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373812+0000","last_undegraded":"2024-05-27T12:36:06.373812+0000","last_fullsized":"2024-05-27T12:36:06.373812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376671+0000","last_change":"2024-05-27T12:36:06.376671+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376671+0000","last_undegraded":"2024-05-27T12:36:06.376671+0000","last_fullsized":"2024-05-27T12:36:06.376671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381990+0000","last_change":"2024-05-27T12:36:06.381990+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381990+0000","last_undegraded":"2024-05-27T12:36:06.381990+0000","last_fullsized":"2024-05-27T12:36:06.381990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380336+0000","last_change":"2024-05-27T12:36:06.380336+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380336+0000","last_undegraded":"2024-05-27T12:36:06.380336+0000","last_fullsized":"2024-05-27T12:36:06.380336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382812+0000","last_change":"2024-05-27T12:36:06.382812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382812+0000","last_undegraded":"2024-05-27T12:36:06.382812+0000","last_fullsized":"2024-05-27T12:36:06.382812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380974+0000","last_change":"2024-05-27T12:36:06.380974+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380974+0000","last_undegraded":"2024-05-27T12:36:06.380974+0000","last_fullsized":"2024-05-27T12:36:06.380974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381165+0000","last_change":"2024-05-27T12:36:06.381165+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381165+0000","last_undegraded":"2024-05-27T12:36:06.381165+0000","last_fullsized":"2024-05-27T12:36:06.381165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384415+0000","last_change":"2024-05-27T12:36:06.384415+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384415+0000","last_undegraded":"2024-05-27T12:36:06.384415+0000","last_fullsized":"2024-05-27T12:36:06.384415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378202+0000","last_change":"2024-05-27T12:36:06.378202+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378202+0000","last_undegraded":"2024-05-27T12:36:06.378202+0000","last_fullsized":"2024-05-27T12:36:06.378202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387285+0000","last_change":"2024-05-27T12:36:06.387285+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387285+0000","last_undegraded":"2024-05-27T12:36:06.387285+0000","last_fullsized":"2024-05-27T12:36:06.387285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373007+0000","last_change":"2024-05-27T12:36:06.373007+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373007+0000","last_undegraded":"2024-05-27T12:36:06.373007+0000","last_fullsized":"2024-05-27T12:36:06.373007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402763+0000","last_change":"2024-05-27T12:36:06.399603+0000","last_active":"2024-05-27T12:36:06.402763+0000","last_peered":"2024-05-27T12:36:06.402763+0000","last_clean":"2024-05-27T12:36:06.402763+0000","last_became_active":"2024-05-27T12:36:06.399286+0000","last_became_peered":"2024-05-27T12:36:06.399286+0000","last_unstale":"2024-05-27T12:36:06.402763+0000","last_undegraded":"2024-05-27T12:36:06.402763+0000","last_fullsized":"2024-05-27T12:36:06.402763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377354+0000","last_change":"2024-05-27T12:36:06.377354+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377354+0000","last_undegraded":"2024-05-27T12:36:06.377354+0000","last_fullsized":"2024-05-27T12:36:06.377354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399712+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.399712+0000","last_peered":"2024-05-27T12:36:06.399712+0000","last_clean":"2024-05-27T12:36:06.399712+0000","last_became_active":"2024-05-27T12:36:06.398355+0000","last_became_peered":"2024-05-27T12:36:06.398355+0000","last_unstale":"2024-05-27T12:36:06.399712+0000","last_undegraded":"2024-05-27T12:36:06.399712+0000","last_fullsized":"2024-05-27T12:36:06.399712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373152+0000","last_change":"2024-05-27T12:36:06.373152+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373152+0000","last_undegraded":"2024-05-27T12:36:06.373152+0000","last_fullsized":"2024-05-27T12:36:06.373152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395682+0000","last_change":"2024-05-27T12:36:06.393410+0000","last_active":"2024-05-27T12:36:06.395682+0000","last_peered":"2024-05-27T12:36:06.395682+0000","last_clean":"2024-05-27T12:36:06.395682+0000","last_became_active":"2024-05-27T12:36:06.392937+0000","last_became_peered":"2024-05-27T12:36:06.392937+0000","last_unstale":"2024-05-27T12:36:06.395682+0000","last_undegraded":"2024-05-27T12:36:06.395682+0000","last_fullsized":"2024-05-27T12:36:06.395682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382381+0000","last_change":"2024-05-27T12:36:06.382381+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382381+0000","last_undegraded":"2024-05-27T12:36:06.382381+0000","last_fullsized":"2024-05-27T12:36:06.382381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.398830+0000","last_change":"2024-05-27T12:36:06.395813+0000","last_active":"2024-05-27T12:36:06.398830+0000","last_peered":"2024-05-27T12:36:06.398830+0000","last_clean":"2024-05-27T12:36:06.398830+0000","last_became_active":"2024-05-27T12:36:06.392355+0000","last_became_peered":"2024-05-27T12:36:06.392355+0000","last_unstale":"2024-05-27T12:36:06.398830+0000","last_undegraded":"2024-05-27T12:36:06.398830+0000","last_fullsized":"2024-05-27T12:36:06.398830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383806+0000","last_change":"2024-05-27T12:36:06.383806+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383806+0000","last_undegraded":"2024-05-27T12:36:06.383806+0000","last_fullsized":"2024-05-27T12:36:06.383806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402620+0000","last_change":"2024-05-27T12:36:06.394640+0000","last_active":"2024-05-27T12:36:06.402620+0000","last_peered":"2024-05-27T12:36:06.402620+0000","last_clean":"2024-05-27T12:36:06.402620+0000","last_became_active":"2024-05-27T12:36:06.394297+0000","last_became_peered":"2024-05-27T12:36:06.394297+0000","last_unstale":"2024-05-27T12:36:06.402620+0000","last_undegraded":"2024-05-27T12:36:06.402620+0000","last_fullsized":"2024-05-27T12:36:06.402620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:17.713278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382750+0000","last_change":"2024-05-27T12:36:06.382750+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382750+0000","last_undegraded":"2024-05-27T12:36:06.382750+0000","last_fullsized":"2024-05-27T12:36:06.382750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402477+0000","last_change":"2024-05-27T12:36:06.394969+0000","last_active":"2024-05-27T12:36:06.402477+0000","last_peered":"2024-05-27T12:36:06.402477+0000","last_clean":"2024-05-27T12:36:06.402477+0000","last_became_active":"2024-05-27T12:36:06.394600+0000","last_became_peered":"2024-05-27T12:36:06.394600+0000","last_unstale":"2024-05-27T12:36:06.402477+0000","last_undegraded":"2024-05-27T12:36:06.402477+0000","last_fullsized":"2024-05-27T12:36:06.402477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:48.343915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376119+0000","last_change":"2024-05-27T12:36:06.376119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376119+0000","last_undegraded":"2024-05-27T12:36:06.376119+0000","last_fullsized":"2024-05-27T12:36:06.376119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.385609+0000","last_change":"2024-05-27T12:36:06.383317+0000","last_active":"2024-05-27T12:36:06.385609+0000","last_peered":"2024-05-27T12:36:06.385609+0000","last_clean":"2024-05-27T12:36:06.385609+0000","last_became_active":"2024-05-27T12:36:06.383006+0000","last_became_peered":"2024-05-27T12:36:06.383006+0000","last_unstale":"2024-05-27T12:36:06.385609+0000","last_undegraded":"2024-05-27T12:36:06.385609+0000","last_fullsized":"2024-05-27T12:36:06.385609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374894+0000","last_change":"2024-05-27T12:36:06.374894+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374894+0000","last_undegraded":"2024-05-27T12:36:06.374894+0000","last_fullsized":"2024-05-27T12:36:06.374894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402610+0000","last_change":"2024-05-27T12:36:06.398382+0000","last_active":"2024-05-27T12:36:06.402610+0000","last_peered":"2024-05-27T12:36:06.402610+0000","last_clean":"2024-05-27T12:36:06.402610+0000","last_became_active":"2024-05-27T12:36:06.396324+0000","last_became_peered":"2024-05-27T12:36:06.396324+0000","last_unstale":"2024-05-27T12:36:06.402610+0000","last_undegraded":"2024-05-27T12:36:06.402610+0000","last_fullsized":"2024-05-27T12:36:06.402610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:50:46.342036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384618+0000","last_change":"2024-05-27T12:36:06.384618+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384618+0000","last_undegraded":"2024-05-27T12:36:06.384618+0000","last_fullsized":"2024-05-27T12:36:06.384618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401315+0000","last_change":"2024-05-27T12:36:06.400135+0000","last_active":"2024-05-27T12:36:06.401315+0000","last_peered":"2024-05-27T12:36:06.401315+0000","last_clean":"2024-05-27T12:36:06.401315+0000","last_became_active":"2024-05-27T12:36:06.399910+0000","last_became_peered":"2024-05-27T12:36:06.399910+0000","last_unstale":"2024-05-27T12:36:06.401315+0000","last_undegraded":"2024-05-27T12:36:06.401315+0000","last_fullsized":"2024-05-27T12:36:06.401315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:45:46.769699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376474+0000","last_change":"2024-05-27T12:36:06.376474+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376474+0000","last_undegraded":"2024-05-27T12:36:06.376474+0000","last_fullsized":"2024-05-27T12:36:06.376474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.386818+0000","last_change":"2024-05-27T12:36:06.383845+0000","last_active":"2024-05-27T12:36:06.386818+0000","last_peered":"2024-05-27T12:36:06.386818+0000","last_clean":"2024-05-27T12:36:06.386818+0000","last_became_active":"2024-05-27T12:36:06.383394+0000","last_became_peered":"2024-05-27T12:36:06.383394+0000","last_unstale":"2024-05-27T12:36:06.386818+0000","last_undegraded":"2024-05-27T12:36:06.386818+0000","last_fullsized":"2024-05-27T12:36:06.386818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373233+0000","last_change":"2024-05-27T12:36:06.373233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373233+0000","last_undegraded":"2024-05-27T12:36:06.373233+0000","last_fullsized":"2024-05-27T12:36:06.373233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402769+0000","last_change":"2024-05-27T12:36:06.398488+0000","last_active":"2024-05-27T12:36:06.402769+0000","last_peered":"2024-05-27T12:36:06.402769+0000","last_clean":"2024-05-27T12:36:06.402769+0000","last_became_active":"2024-05-27T12:36:06.397511+0000","last_became_peered":"2024-05-27T12:36:06.397511+0000","last_unstale":"2024-05-27T12:36:06.402769+0000","last_undegraded":"2024-05-27T12:36:06.402769+0000","last_fullsized":"2024-05-27T12:36:06.402769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:40.771809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379981+0000","last_change":"2024-05-27T12:36:06.379981+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379981+0000","last_undegraded":"2024-05-27T12:36:06.379981+0000","last_fullsized":"2024-05-27T12:36:06.379981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397658+0000","last_change":"2024-05-27T12:36:06.393888+0000","last_active":"2024-05-27T12:36:06.397658+0000","last_peered":"2024-05-27T12:36:06.397658+0000","last_clean":"2024-05-27T12:36:06.397658+0000","last_became_active":"2024-05-27T12:36:06.393567+0000","last_became_peered":"2024-05-27T12:36:06.393567+0000","last_unstale":"2024-05-27T12:36:06.397658+0000","last_undegraded":"2024-05-27T12:36:06.397658+0000","last_fullsized":"2024-05-27T12:36:06.397658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374467+0000","last_change":"2024-05-27T12:36:06.374467+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374467+0000","last_undegraded":"2024-05-27T12:36:06.374467+0000","last_fullsized":"2024-05-27T12:36:06.374467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402643+0000","last_change":"2024-05-27T12:36:06.396297+0000","last_active":"2024-05-27T12:36:06.402643+0000","last_peered":"2024-05-27T12:36:06.402643+0000","last_clean":"2024-05-27T12:36:06.402643+0000","last_became_active":"2024-05-27T12:36:06.395605+0000","last_became_peered":"2024-05-27T12:36:06.395605+0000","last_unstale":"2024-05-27T12:36:06.402643+0000","last_undegraded":"2024-05-27T12:36:06.402643+0000","last_fullsized":"2024-05-27T12:36:06.402643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:04.614416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373670+0000","last_change":"2024-05-27T12:36:06.373670+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373670+0000","last_undegraded":"2024-05-27T12:36:06.373670+0000","last_fullsized":"2024-05-27T12:36:06.373670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396083+0000","last_change":"2024-05-27T12:36:06.391699+0000","last_active":"2024-05-27T12:36:06.396083+0000","last_peered":"2024-05-27T12:36:06.396083+0000","last_clean":"2024-05-27T12:36:06.396083+0000","last_became_active":"2024-05-27T12:36:06.391285+0000","last_became_peered":"2024-05-27T12:36:06.391285+0000","last_unstale":"2024-05-27T12:36:06.396083+0000","last_undegraded":"2024-05-27T12:36:06.396083+0000","last_fullsized":"2024-05-27T12:36:06.396083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386703+0000","last_change":"2024-05-27T12:36:06.386703+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386703+0000","last_undegraded":"2024-05-27T12:36:06.386703+0000","last_fullsized":"2024-05-27T12:36:06.386703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400607+0000","last_change":"2024-05-27T12:36:06.399046+0000","last_active":"2024-05-27T12:36:06.400607+0000","last_peered":"2024-05-27T12:36:06.400607+0000","last_clean":"2024-05-27T12:36:06.400607+0000","last_became_active":"2024-05-27T12:36:06.396837+0000","last_became_peered":"2024-05-27T12:36:06.396837+0000","last_unstale":"2024-05-27T12:36:06.400607+0000","last_undegraded":"2024-05-27T12:36:06.400607+0000","last_fullsized":"2024-05-27T12:36:06.400607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376569+0000","last_change":"2024-05-27T12:36:06.376569+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376569+0000","last_undegraded":"2024-05-27T12:36:06.376569+0000","last_fullsized":"2024-05-27T12:36:06.376569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400216+0000","last_change":"2024-05-27T12:36:06.397002+0000","last_active":"2024-05-27T12:36:06.400216+0000","last_peered":"2024-05-27T12:36:06.400216+0000","last_clean":"2024-05-27T12:36:06.400216+0000","last_became_active":"2024-05-27T12:36:06.396513+0000","last_became_peered":"2024-05-27T12:36:06.396513+0000","last_unstale":"2024-05-27T12:36:06.400216+0000","last_undegraded":"2024-05-27T12:36:06.400216+0000","last_fullsized":"2024-05-27T12:36:06.400216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382122+0000","last_change":"2024-05-27T12:36:06.382122+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382122+0000","last_undegraded":"2024-05-27T12:36:06.382122+0000","last_fullsized":"2024-05-27T12:36:06.382122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395817+0000","last_change":"2024-05-27T12:36:06.390945+0000","last_active":"2024-05-27T12:36:06.395817+0000","last_peered":"2024-05-27T12:36:06.395817+0000","last_clean":"2024-05-27T12:36:06.395817+0000","last_became_active":"2024-05-27T12:36:06.390575+0000","last_became_peered":"2024-05-27T12:36:06.390575+0000","last_unstale":"2024-05-27T12:36:06.395817+0000","last_undegraded":"2024-05-27T12:36:06.395817+0000","last_fullsized":"2024-05-27T12:36:06.395817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384033+0000","last_change":"2024-05-27T12:36:06.384033+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384033+0000","last_undegraded":"2024-05-27T12:36:06.384033+0000","last_fullsized":"2024-05-27T12:36:06.384033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402551+0000","last_change":"2024-05-27T12:36:06.396687+0000","last_active":"2024-05-27T12:36:06.402551+0000","last_peered":"2024-05-27T12:36:06.402551+0000","last_clean":"2024-05-27T12:36:06.402551+0000","last_became_active":"2024-05-27T12:36:06.395340+0000","last_became_peered":"2024-05-27T12:36:06.395340+0000","last_unstale":"2024-05-27T12:36:06.402551+0000","last_undegraded":"2024-05-27T12:36:06.402551+0000","last_fullsized":"2024-05-27T12:36:06.402551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:14.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373578+0000","last_change":"2024-05-27T12:36:06.373578+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373578+0000","last_undegraded":"2024-05-27T12:36:06.373578+0000","last_fullsized":"2024-05-27T12:36:06.373578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399923+0000","last_change":"2024-05-27T12:36:06.393791+0000","last_active":"2024-05-27T12:36:06.399923+0000","last_peered":"2024-05-27T12:36:06.399923+0000","last_clean":"2024-05-27T12:36:06.399923+0000","last_became_active":"2024-05-27T12:36:06.391768+0000","last_became_peered":"2024-05-27T12:36:06.391768+0000","last_unstale":"2024-05-27T12:36:06.399923+0000","last_undegraded":"2024-05-27T12:36:06.399923+0000","last_fullsized":"2024-05-27T12:36:06.399923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:11:10.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380582+0000","last_change":"2024-05-27T12:36:06.380582+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380582+0000","last_undegraded":"2024-05-27T12:36:06.380582+0000","last_fullsized":"2024-05-27T12:36:06.380582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396504+0000","last_change":"2024-05-27T12:36:06.388667+0000","last_active":"2024-05-27T12:36:06.396504+0000","last_peered":"2024-05-27T12:36:06.396504+0000","last_clean":"2024-05-27T12:36:06.396504+0000","last_became_active":"2024-05-27T12:36:06.388369+0000","last_became_peered":"2024-05-27T12:36:06.388369+0000","last_unstale":"2024-05-27T12:36:06.396504+0000","last_undegraded":"2024-05-27T12:36:06.396504+0000","last_fullsized":"2024-05-27T12:36:06.396504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:49.907565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374535+0000","last_change":"2024-05-27T12:36:06.374535+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374535+0000","last_undegraded":"2024-05-27T12:36:06.374535+0000","last_fullsized":"2024-05-27T12:36:06.374535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394967+0000","last_change":"2024-05-27T12:36:06.391794+0000","last_active":"2024-05-27T12:36:06.394967+0000","last_peered":"2024-05-27T12:36:06.394967+0000","last_clean":"2024-05-27T12:36:06.394967+0000","last_became_active":"2024-05-27T12:36:06.389977+0000","last_became_peered":"2024-05-27T12:36:06.389977+0000","last_unstale":"2024-05-27T12:36:06.394967+0000","last_undegraded":"2024-05-27T12:36:06.394967+0000","last_fullsized":"2024-05-27T12:36:06.394967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373084+0000","last_change":"2024-05-27T12:36:06.373084+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373084+0000","last_undegraded":"2024-05-27T12:36:06.373084+0000","last_fullsized":"2024-05-27T12:36:06.373084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397674+0000","last_change":"2024-05-27T12:36:06.393310+0000","last_active":"2024-05-27T12:36:06.397674+0000","last_peered":"2024-05-27T12:36:06.397674+0000","last_clean":"2024-05-27T12:36:06.397674+0000","last_became_active":"2024-05-27T12:36:06.392014+0000","last_became_peered":"2024-05-27T12:36:06.392014+0000","last_unstale":"2024-05-27T12:36:06.397674+0000","last_undegraded":"2024-05-27T12:36:06.397674+0000","last_fullsized":"2024-05-27T12:36:06.397674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374186+0000","last_change":"2024-05-27T12:36:06.374186+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374186+0000","last_undegraded":"2024-05-27T12:36:06.374186+0000","last_fullsized":"2024-05-27T12:36:06.374186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399777+0000","last_change":"2024-05-27T12:36:06.393400+0000","last_active":"2024-05-27T12:36:06.399777+0000","last_peered":"2024-05-27T12:36:06.399777+0000","last_clean":"2024-05-27T12:36:06.399777+0000","last_became_active":"2024-05-27T12:36:06.392996+0000","last_became_peered":"2024-05-27T12:36:06.392996+0000","last_unstale":"2024-05-27T12:36:06.399777+0000","last_undegraded":"2024-05-27T12:36:06.399777+0000","last_fullsized":"2024-05-27T12:36:06.399777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386244+0000","last_change":"2024-05-27T12:36:06.386244+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386244+0000","last_undegraded":"2024-05-27T12:36:06.386244+0000","last_fullsized":"2024-05-27T12:36:06.386244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400560+0000","last_change":"2024-05-27T12:36:06.396022+0000","last_active":"2024-05-27T12:36:06.400560+0000","last_peered":"2024-05-27T12:36:06.400560+0000","last_clean":"2024-05-27T12:36:06.400560+0000","last_became_active":"2024-05-27T12:36:06.395727+0000","last_became_peered":"2024-05-27T12:36:06.395727+0000","last_unstale":"2024-05-27T12:36:06.400560+0000","last_undegraded":"2024-05-27T12:36:06.400560+0000","last_fullsized":"2024-05-27T12:36:06.400560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:03.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379447+0000","last_change":"2024-05-27T12:36:06.379447+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379447+0000","last_undegraded":"2024-05-27T12:36:06.379447+0000","last_fullsized":"2024-05-27T12:36:06.379447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395838+0000","last_change":"2024-05-27T12:36:06.393032+0000","last_active":"2024-05-27T12:36:06.395838+0000","last_peered":"2024-05-27T12:36:06.395838+0000","last_clean":"2024-05-27T12:36:06.395838+0000","last_became_active":"2024-05-27T12:36:06.392754+0000","last_became_peered":"2024-05-27T12:36:06.392754+0000","last_unstale":"2024-05-27T12:36:06.395838+0000","last_undegraded":"2024-05-27T12:36:06.395838+0000","last_fullsized":"2024-05-27T12:36:06.395838+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379855+0000","last_change":"2024-05-27T12:36:06.379855+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379855+0000","last_undegraded":"2024-05-27T12:36:06.379855+0000","last_fullsized":"2024-05-27T12:36:06.379855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401022+0000","last_change":"2024-05-27T12:36:06.399546+0000","last_active":"2024-05-27T12:36:06.401022+0000","last_peered":"2024-05-27T12:36:06.401022+0000","last_clean":"2024-05-27T12:36:06.401022+0000","last_became_active":"2024-05-27T12:36:06.399228+0000","last_became_peered":"2024-05-27T12:36:06.399228+0000","last_unstale":"2024-05-27T12:36:06.401022+0000","last_undegraded":"2024-05-27T12:36:06.401022+0000","last_fullsized":"2024-05-27T12:36:06.401022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375413+0000","last_change":"2024-05-27T12:36:06.375413+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375413+0000","last_undegraded":"2024-05-27T12:36:06.375413+0000","last_fullsized":"2024-05-27T12:36:06.375413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395586+0000","last_change":"2024-05-27T12:36:06.391239+0000","last_active":"2024-05-27T12:36:06.395586+0000","last_peered":"2024-05-27T12:36:06.395586+0000","last_clean":"2024-05-27T12:36:06.395586+0000","last_became_active":"2024-05-27T12:36:06.390800+0000","last_became_peered":"2024-05-27T12:36:06.390800+0000","last_unstale":"2024-05-27T12:36:06.395586+0000","last_undegraded":"2024-05-27T12:36:06.395586+0000","last_fullsized":"2024-05-27T12:36:06.395586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375292+0000","last_change":"2024-05-27T12:36:06.375292+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375292+0000","last_undegraded":"2024-05-27T12:36:06.375292+0000","last_fullsized":"2024-05-27T12:36:06.375292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.391355+0000","last_change":"2024-05-27T12:36:06.389001+0000","last_active":"2024-05-27T12:36:06.391355+0000","last_peered":"2024-05-27T12:36:06.391355+0000","last_clean":"2024-05-27T12:36:06.391355+0000","last_became_active":"2024-05-27T12:36:06.388700+0000","last_became_peered":"2024-05-27T12:36:06.388700+0000","last_unstale":"2024-05-27T12:36:06.391355+0000","last_undegraded":"2024-05-27T12:36:06.391355+0000","last_fullsized":"2024-05-27T12:36:06.391355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.372335+0000","last_change":"2024-05-27T12:36:06.372335+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.372335+0000","last_undegraded":"2024-05-27T12:36:06.372335+0000","last_fullsized":"2024-05-27T12:36:06.372335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394265+0000","last_change":"2024-05-27T12:36:06.390271+0000","last_active":"2024-05-27T12:36:06.394265+0000","last_peered":"2024-05-27T12:36:06.394265+0000","last_clean":"2024-05-27T12:36:06.394265+0000","last_became_active":"2024-05-27T12:36:06.389933+0000","last_became_peered":"2024-05-27T12:36:06.389933+0000","last_unstale":"2024-05-27T12:36:06.394265+0000","last_undegraded":"2024-05-27T12:36:06.394265+0000","last_fullsized":"2024-05-27T12:36:06.394265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378119+0000","last_change":"2024-05-27T12:36:06.378119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378119+0000","last_undegraded":"2024-05-27T12:36:06.378119+0000","last_fullsized":"2024-05-27T12:36:06.378119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396869+0000","last_change":"2024-05-27T12:36:06.389847+0000","last_active":"2024-05-27T12:36:06.396869+0000","last_peered":"2024-05-27T12:36:06.396869+0000","last_clean":"2024-05-27T12:36:06.396869+0000","last_became_active":"2024-05-27T12:36:06.389431+0000","last_became_peered":"2024-05-27T12:36:06.389431+0000","last_unstale":"2024-05-27T12:36:06.396869+0000","last_undegraded":"2024-05-27T12:36:06.396869+0000","last_fullsized":"2024-05-27T12:36:06.396869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:54:21.911265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373815+0000","last_change":"2024-05-27T12:36:06.373815+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373815+0000","last_undegraded":"2024-05-27T12:36:06.373815+0000","last_fullsized":"2024-05-27T12:36:06.373815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400629+0000","last_change":"2024-05-27T12:36:06.399243+0000","last_active":"2024-05-27T12:36:06.400629+0000","last_peered":"2024-05-27T12:36:06.400629+0000","last_clean":"2024-05-27T12:36:06.400629+0000","last_became_active":"2024-05-27T12:36:06.397252+0000","last_became_peered":"2024-05-27T12:36:06.397252+0000","last_unstale":"2024-05-27T12:36:06.400629+0000","last_undegraded":"2024-05-27T12:36:06.400629+0000","last_fullsized":"2024-05-27T12:36:06.400629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:38.949294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383963+0000","last_change":"2024-05-27T12:36:06.383963+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383963+0000","last_undegraded":"2024-05-27T12:36:06.383963+0000","last_fullsized":"2024-05-27T12:36:06.383963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400637+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.400637+0000","last_peered":"2024-05-27T12:36:06.400637+0000","last_clean":"2024-05-27T12:36:06.400637+0000","last_became_active":"2024-05-27T12:36:06.398205+0000","last_became_peered":"2024-05-27T12:36:06.398205+0000","last_unstale":"2024-05-27T12:36:06.400637+0000","last_undegraded":"2024-05-27T12:36:06.400637+0000","last_fullsized":"2024-05-27T12:36:06.400637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.393078+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:36:06.393078+0000","last_peered":"2024-05-27T12:36:06.393078+0000","last_clean":"2024-05-27T12:36:06.393078+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:36:06.393078+0000","last_undegraded":"2024-05-27T12:36:06.393078+0000","last_fullsized":"2024-05-27T12:36:06.393078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:36:12.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:36:12.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.577+0000 7f1967e00640 1 Processor -- start 2024-05-27T12:36:12.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.577+0000 7f1967e00640 1 -- start start 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f196806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1968154b20 0x7f196815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 0x7f196815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f196815b940 con 0x7f1968154b20 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f196815bc20 con 0x7f1968154fa0 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f1967e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f196806de60 con 0x7f196806fe50 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f195fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1968154b20 0x7f196815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f195fe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1968154b20 0x7f196815acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46232/0 (socket says 172.21.15.94:46232) 2024-05-27T12:36:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.579+0000 7f195fe00640 1 -- 172.21.15.94:0/2423894924 learned_addr learned my addr 172.21.15.94:0/2423894924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f1967400640 1 --2- 172.21.15.94:0/2423894924 >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 0x7f196815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f1966a00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f196806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f195fe00640 1 -- 172.21.15.94:0/2423894924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 msgr2=0x7f196806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f195fe00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f196806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f195fe00640 1 -- 172.21.15.94:0/2423894924 >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 msgr2=0x7f196815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f195fe00640 1 --2- 172.21.15.94:0/2423894924 >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 0x7f196815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f195fe00640 1 -- 172.21.15.94:0/2423894924 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196806e140 con 0x7f1968154b20 2024-05-27T12:36:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.580+0000 7f1967400640 1 --2- 172.21.15.94:0/2423894924 >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 0x7f196815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:12.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.581+0000 7f195fe00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154b20 0x7f196815acc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19680cd210 tx=0x7f195002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fcddd6229ddcd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.581+0000 7f1966000640 1 -- 172.21.15.94:0/2423894924 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950037070 con 0x7f1968154b20 2024-05-27T12:36:12.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.581+0000 7f1966000640 1 -- 172.21.15.94:0/2423894924 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1950032440 con 0x7f1968154b20 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.581+0000 7f1966000640 1 -- 172.21.15.94:0/2423894924 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950032810 con 0x7f1968154b20 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 -- 172.21.15.94:0/2423894924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154b20 msgr2=0x7f196815acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154b20 0x7f196815acc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19680cd210 tx=0x7f195002a1e0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 -- 172.21.15.94:0/2423894924 shutdown_connections 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 --2- 172.21.15.94:0/2423894924 >> v2:172.21.15.97:3300/0 conn(0x7f1968154fa0 0x7f196815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154b20 0x7f196815acc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 --2- 172.21.15.94:0/2423894924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f196806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 -- 172.21.15.94:0/2423894924 >> 172.21.15.94:0/2423894924 conn(0x7f19680b2160 msgr2=0x7f19680b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 -- 172.21.15.94:0/2423894924 shutdown_connections 2024-05-27T12:36:12.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.582+0000 7f1967e00640 1 -- 172.21.15.94:0/2423894924 wait complete. 2024-05-27T12:36:12.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.584+0000 7f1967e00640 1 Processor -- start 2024-05-27T12:36:12.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.584+0000 7f1967e00640 1 -- start start 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f1968153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 0x7f19680aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f196806ee80 con 0x7f1968154fa0 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19680b4e10 con 0x7f1968154b20 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f1968140360 con 0x7f196806fe50 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46242/0 (socket says 172.21.15.94:46242) 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f1967400640 1 -- 172.21.15.94:0/1800789940 learned_addr learned my addr 172.21.15.94:0/1800789940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.585+0000 7f195fe00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 0x7f19680aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1966a00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f1968153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 -- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 msgr2=0x7f1968153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f1968153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 -- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 msgr2=0x7f19680aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 0x7f19680aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 -- 172.21.15.94:0/1800789940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19680af7e0 con 0x7f1968154fa0 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f195fe00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 0x7f19680aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1966a00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f1968153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967400640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196000cdb0 tx=0x7f196000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196000f680 con 0x7f1968154fa0 2024-05-27T12:36:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196000f840 con 0x7f1968154fa0 2024-05-27T12:36:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196000fc30 con 0x7f1968154fa0 2024-05-27T12:36:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19680af220 con 0x7f1968154fa0 2024-05-27T12:36:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.586+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19680a9530 con 0x7f1968154fa0 2024-05-27T12:36:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.587+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1968075460 con 0x7f1968154fa0 2024-05-27T12:36:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.588+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f196000e3e0 con 0x7f1968154fa0 2024-05-27T12:36:12.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.590+0000 7f195ea00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 0x7f194406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:12.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.590+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f196008c8a0 con 0x7f1968154fa0 2024-05-27T12:36:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.590+0000 7f1966a00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 0x7f194406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:12.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.592+0000 7f1966a00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 0x7f194406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1958008890 tx=0x7f1958008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:12.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.594+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1960014030 con 0x7f1968154fa0 2024-05-27T12:36:12.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.785+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19680ce6d0 con 0x7f194406c8f0 2024-05-27T12:36:12.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.803+0000 7f195ea00640 1 -- 172.21.15.94:0/1800789940 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f19680ce6d0 con 0x7f194406c8f0 2024-05-27T12:36:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:12.807 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 msgr2=0x7f194406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 0x7f194406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1958008890 tx=0x7f1958008040 comp rx=0 tx=0).stop 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 msgr2=0x7f196813fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196000cdb0 tx=0x7f196000a650 comp rx=0 tx=0).stop 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 shutdown_connections 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f194406c8f0 0x7f194406edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968154b20 0x7f19680aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1968154fa0 0x7f196813fe20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 --2- 172.21.15.94:0/1800789940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f196806fe50 0x7f1968153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 >> 172.21.15.94:0/1800789940 conn(0x7f19680b2160 msgr2=0x7f1968071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 shutdown_connections 2024-05-27T12:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:12.808+0000 7f1967e00640 1 -- 172.21.15.94:0/1800789940 wait complete. 2024-05-27T12:36:12.825 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-27T12:36:10.883672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3948544,"data_stored":2236530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536186"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380979+0000","last_change":"2024-05-27T12:36:06.380979+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380979+0000","last_undegraded":"2024-05-27T12:36:06.380979+0000","last_fullsized":"2024-05-27T12:36:06.380979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381776+0000","last_change":"2024-05-27T12:36:06.381776+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381776+0000","last_undegraded":"2024-05-27T12:36:06.381776+0000","last_fullsized":"2024-05-27T12:36:06.381776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379465+0000","last_change":"2024-05-27T12:36:06.379465+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379465+0000","last_undegraded":"2024-05-27T12:36:06.379465+0000","last_fullsized":"2024-05-27T12:36:06.379465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374883+0000","last_change":"2024-05-27T12:36:06.374883+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374883+0000","last_undegraded":"2024-05-27T12:36:06.374883+0000","last_fullsized":"2024-05-27T12:36:06.374883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374479+0000","last_change":"2024-05-27T12:36:06.374479+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374479+0000","last_undegraded":"2024-05-27T12:36:06.374479+0000","last_fullsized":"2024-05-27T12:36:06.374479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377579+0000","last_change":"2024-05-27T12:36:06.377579+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377579+0000","last_undegraded":"2024-05-27T12:36:06.377579+0000","last_fullsized":"2024-05-27T12:36:06.377579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.385452+0000","last_change":"2024-05-27T12:36:06.385452+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.385452+0000","last_undegraded":"2024-05-27T12:36:06.385452+0000","last_fullsized":"2024-05-27T12:36:06.385452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387969+0000","last_change":"2024-05-27T12:36:06.387969+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387969+0000","last_undegraded":"2024-05-27T12:36:06.387969+0000","last_fullsized":"2024-05-27T12:36:06.387969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382637+0000","last_change":"2024-05-27T12:36:06.382637+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382637+0000","last_undegraded":"2024-05-27T12:36:06.382637+0000","last_fullsized":"2024-05-27T12:36:06.382637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379941+0000","last_change":"2024-05-27T12:36:06.379941+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379941+0000","last_undegraded":"2024-05-27T12:36:06.379941+0000","last_fullsized":"2024-05-27T12:36:06.379941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.389103+0000","last_change":"2024-05-27T12:36:06.389103+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.389103+0000","last_undegraded":"2024-05-27T12:36:06.389103+0000","last_fullsized":"2024-05-27T12:36:06.389103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381444+0000","last_change":"2024-05-27T12:36:06.381444+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381444+0000","last_undegraded":"2024-05-27T12:36:06.381444+0000","last_fullsized":"2024-05-27T12:36:06.381444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373801+0000","last_change":"2024-05-27T12:36:06.373801+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373801+0000","last_undegraded":"2024-05-27T12:36:06.373801+0000","last_fullsized":"2024-05-27T12:36:06.373801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376906+0000","last_change":"2024-05-27T12:36:06.376906+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376906+0000","last_undegraded":"2024-05-27T12:36:06.376906+0000","last_fullsized":"2024-05-27T12:36:06.376906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377394+0000","last_change":"2024-05-27T12:36:06.377394+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377394+0000","last_undegraded":"2024-05-27T12:36:06.377394+0000","last_fullsized":"2024-05-27T12:36:06.377394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384646+0000","last_change":"2024-05-27T12:36:06.384646+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384646+0000","last_undegraded":"2024-05-27T12:36:06.384646+0000","last_fullsized":"2024-05-27T12:36:06.384646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.391236+0000","last_change":"2024-05-27T12:36:06.391236+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.391236+0000","last_undegraded":"2024-05-27T12:36:06.391236+0000","last_fullsized":"2024-05-27T12:36:06.391236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381233+0000","last_change":"2024-05-27T12:36:06.381233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381233+0000","last_undegraded":"2024-05-27T12:36:06.381233+0000","last_fullsized":"2024-05-27T12:36:06.381233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376290+0000","last_change":"2024-05-27T12:36:06.376290+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376290+0000","last_undegraded":"2024-05-27T12:36:06.376290+0000","last_fullsized":"2024-05-27T12:36:06.376290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376387+0000","last_change":"2024-05-27T12:36:06.376387+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376387+0000","last_undegraded":"2024-05-27T12:36:06.376387+0000","last_fullsized":"2024-05-27T12:36:06.376387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381597+0000","last_change":"2024-05-27T12:36:06.381597+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381597+0000","last_undegraded":"2024-05-27T12:36:06.381597+0000","last_fullsized":"2024-05-27T12:36:06.381597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384281+0000","last_change":"2024-05-27T12:36:06.384281+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384281+0000","last_undegraded":"2024-05-27T12:36:06.384281+0000","last_fullsized":"2024-05-27T12:36:06.384281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373812+0000","last_change":"2024-05-27T12:36:06.373812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373812+0000","last_undegraded":"2024-05-27T12:36:06.373812+0000","last_fullsized":"2024-05-27T12:36:06.373812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376671+0000","last_change":"2024-05-27T12:36:06.376671+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376671+0000","last_undegraded":"2024-05-27T12:36:06.376671+0000","last_fullsized":"2024-05-27T12:36:06.376671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381990+0000","last_change":"2024-05-27T12:36:06.381990+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381990+0000","last_undegraded":"2024-05-27T12:36:06.381990+0000","last_fullsized":"2024-05-27T12:36:06.381990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380336+0000","last_change":"2024-05-27T12:36:06.380336+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380336+0000","last_undegraded":"2024-05-27T12:36:06.380336+0000","last_fullsized":"2024-05-27T12:36:06.380336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382812+0000","last_change":"2024-05-27T12:36:06.382812+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382812+0000","last_undegraded":"2024-05-27T12:36:06.382812+0000","last_fullsized":"2024-05-27T12:36:06.382812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380974+0000","last_change":"2024-05-27T12:36:06.380974+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380974+0000","last_undegraded":"2024-05-27T12:36:06.380974+0000","last_fullsized":"2024-05-27T12:36:06.380974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.381165+0000","last_change":"2024-05-27T12:36:06.381165+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.381165+0000","last_undegraded":"2024-05-27T12:36:06.381165+0000","last_fullsized":"2024-05-27T12:36:06.381165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384415+0000","last_change":"2024-05-27T12:36:06.384415+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384415+0000","last_undegraded":"2024-05-27T12:36:06.384415+0000","last_fullsized":"2024-05-27T12:36:06.384415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378202+0000","last_change":"2024-05-27T12:36:06.378202+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378202+0000","last_undegraded":"2024-05-27T12:36:06.378202+0000","last_fullsized":"2024-05-27T12:36:06.378202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.387285+0000","last_change":"2024-05-27T12:36:06.387285+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.387285+0000","last_undegraded":"2024-05-27T12:36:06.387285+0000","last_fullsized":"2024-05-27T12:36:06.387285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373007+0000","last_change":"2024-05-27T12:36:06.373007+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373007+0000","last_undegraded":"2024-05-27T12:36:06.373007+0000","last_fullsized":"2024-05-27T12:36:06.373007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402763+0000","last_change":"2024-05-27T12:36:06.399603+0000","last_active":"2024-05-27T12:36:06.402763+0000","last_peered":"2024-05-27T12:36:06.402763+0000","last_clean":"2024-05-27T12:36:06.402763+0000","last_became_active":"2024-05-27T12:36:06.399286+0000","last_became_peered":"2024-05-27T12:36:06.399286+0000","last_unstale":"2024-05-27T12:36:06.402763+0000","last_undegraded":"2024-05-27T12:36:06.402763+0000","last_fullsized":"2024-05-27T12:36:06.402763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.377354+0000","last_change":"2024-05-27T12:36:06.377354+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.377354+0000","last_undegraded":"2024-05-27T12:36:06.377354+0000","last_fullsized":"2024-05-27T12:36:06.377354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399712+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.399712+0000","last_peered":"2024-05-27T12:36:06.399712+0000","last_clean":"2024-05-27T12:36:06.399712+0000","last_became_active":"2024-05-27T12:36:06.398355+0000","last_became_peered":"2024-05-27T12:36:06.398355+0000","last_unstale":"2024-05-27T12:36:06.399712+0000","last_undegraded":"2024-05-27T12:36:06.399712+0000","last_fullsized":"2024-05-27T12:36:06.399712+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373152+0000","last_change":"2024-05-27T12:36:06.373152+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373152+0000","last_undegraded":"2024-05-27T12:36:06.373152+0000","last_fullsized":"2024-05-27T12:36:06.373152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395682+0000","last_change":"2024-05-27T12:36:06.393410+0000","last_active":"2024-05-27T12:36:06.395682+0000","last_peered":"2024-05-27T12:36:06.395682+0000","last_clean":"2024-05-27T12:36:06.395682+0000","last_became_active":"2024-05-27T12:36:06.392937+0000","last_became_peered":"2024-05-27T12:36:06.392937+0000","last_unstale":"2024-05-27T12:36:06.395682+0000","last_undegraded":"2024-05-27T12:36:06.395682+0000","last_fullsized":"2024-05-27T12:36:06.395682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382381+0000","last_change":"2024-05-27T12:36:06.382381+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382381+0000","last_undegraded":"2024-05-27T12:36:06.382381+0000","last_fullsized":"2024-05-27T12:36:06.382381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.398830+0000","last_change":"2024-05-27T12:36:06.395813+0000","last_active":"2024-05-27T12:36:06.398830+0000","last_peered":"2024-05-27T12:36:06.398830+0000","last_clean":"2024-05-27T12:36:06.398830+0000","last_became_active":"2024-05-27T12:36:06.392355+0000","last_became_peered":"2024-05-27T12:36:06.392355+0000","last_unstale":"2024-05-27T12:36:06.398830+0000","last_undegraded":"2024-05-27T12:36:06.398830+0000","last_fullsized":"2024-05-27T12:36:06.398830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383806+0000","last_change":"2024-05-27T12:36:06.383806+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383806+0000","last_undegraded":"2024-05-27T12:36:06.383806+0000","last_fullsized":"2024-05-27T12:36:06.383806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402620+0000","last_change":"2024-05-27T12:36:06.394640+0000","last_active":"2024-05-27T12:36:06.402620+0000","last_peered":"2024-05-27T12:36:06.402620+0000","last_clean":"2024-05-27T12:36:06.402620+0000","last_became_active":"2024-05-27T12:36:06.394297+0000","last_became_peered":"2024-05-27T12:36:06.394297+0000","last_unstale":"2024-05-27T12:36:06.402620+0000","last_undegraded":"2024-05-27T12:36:06.402620+0000","last_fullsized":"2024-05-27T12:36:06.402620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:17.713278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382750+0000","last_change":"2024-05-27T12:36:06.382750+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382750+0000","last_undegraded":"2024-05-27T12:36:06.382750+0000","last_fullsized":"2024-05-27T12:36:06.382750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402477+0000","last_change":"2024-05-27T12:36:06.394969+0000","last_active":"2024-05-27T12:36:06.402477+0000","last_peered":"2024-05-27T12:36:06.402477+0000","last_clean":"2024-05-27T12:36:06.402477+0000","last_became_active":"2024-05-27T12:36:06.394600+0000","last_became_peered":"2024-05-27T12:36:06.394600+0000","last_unstale":"2024-05-27T12:36:06.402477+0000","last_undegraded":"2024-05-27T12:36:06.402477+0000","last_fullsized":"2024-05-27T12:36:06.402477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:48.343915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376119+0000","last_change":"2024-05-27T12:36:06.376119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376119+0000","last_undegraded":"2024-05-27T12:36:06.376119+0000","last_fullsized":"2024-05-27T12:36:06.376119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.385609+0000","last_change":"2024-05-27T12:36:06.383317+0000","last_active":"2024-05-27T12:36:06.385609+0000","last_peered":"2024-05-27T12:36:06.385609+0000","last_clean":"2024-05-27T12:36:06.385609+0000","last_became_active":"2024-05-27T12:36:06.383006+0000","last_became_peered":"2024-05-27T12:36:06.383006+0000","last_unstale":"2024-05-27T12:36:06.385609+0000","last_undegraded":"2024-05-27T12:36:06.385609+0000","last_fullsized":"2024-05-27T12:36:06.385609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374894+0000","last_change":"2024-05-27T12:36:06.374894+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374894+0000","last_undegraded":"2024-05-27T12:36:06.374894+0000","last_fullsized":"2024-05-27T12:36:06.374894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402610+0000","last_change":"2024-05-27T12:36:06.398382+0000","last_active":"2024-05-27T12:36:06.402610+0000","last_peered":"2024-05-27T12:36:06.402610+0000","last_clean":"2024-05-27T12:36:06.402610+0000","last_became_active":"2024-05-27T12:36:06.396324+0000","last_became_peered":"2024-05-27T12:36:06.396324+0000","last_unstale":"2024-05-27T12:36:06.402610+0000","last_undegraded":"2024-05-27T12:36:06.402610+0000","last_fullsized":"2024-05-27T12:36:06.402610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:50:46.342036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384618+0000","last_change":"2024-05-27T12:36:06.384618+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384618+0000","last_undegraded":"2024-05-27T12:36:06.384618+0000","last_fullsized":"2024-05-27T12:36:06.384618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401315+0000","last_change":"2024-05-27T12:36:06.400135+0000","last_active":"2024-05-27T12:36:06.401315+0000","last_peered":"2024-05-27T12:36:06.401315+0000","last_clean":"2024-05-27T12:36:06.401315+0000","last_became_active":"2024-05-27T12:36:06.399910+0000","last_became_peered":"2024-05-27T12:36:06.399910+0000","last_unstale":"2024-05-27T12:36:06.401315+0000","last_undegraded":"2024-05-27T12:36:06.401315+0000","last_fullsized":"2024-05-27T12:36:06.401315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:45:46.769699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376474+0000","last_change":"2024-05-27T12:36:06.376474+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376474+0000","last_undegraded":"2024-05-27T12:36:06.376474+0000","last_fullsized":"2024-05-27T12:36:06.376474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.386818+0000","last_change":"2024-05-27T12:36:06.383845+0000","last_active":"2024-05-27T12:36:06.386818+0000","last_peered":"2024-05-27T12:36:06.386818+0000","last_clean":"2024-05-27T12:36:06.386818+0000","last_became_active":"2024-05-27T12:36:06.383394+0000","last_became_peered":"2024-05-27T12:36:06.383394+0000","last_unstale":"2024-05-27T12:36:06.386818+0000","last_undegraded":"2024-05-27T12:36:06.386818+0000","last_fullsized":"2024-05-27T12:36:06.386818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373233+0000","last_change":"2024-05-27T12:36:06.373233+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373233+0000","last_undegraded":"2024-05-27T12:36:06.373233+0000","last_fullsized":"2024-05-27T12:36:06.373233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402769+0000","last_change":"2024-05-27T12:36:06.398488+0000","last_active":"2024-05-27T12:36:06.402769+0000","last_peered":"2024-05-27T12:36:06.402769+0000","last_clean":"2024-05-27T12:36:06.402769+0000","last_became_active":"2024-05-27T12:36:06.397511+0000","last_became_peered":"2024-05-27T12:36:06.397511+0000","last_unstale":"2024-05-27T12:36:06.402769+0000","last_undegraded":"2024-05-27T12:36:06.402769+0000","last_fullsized":"2024-05-27T12:36:06.402769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:40.771809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379981+0000","last_change":"2024-05-27T12:36:06.379981+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379981+0000","last_undegraded":"2024-05-27T12:36:06.379981+0000","last_fullsized":"2024-05-27T12:36:06.379981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397658+0000","last_change":"2024-05-27T12:36:06.393888+0000","last_active":"2024-05-27T12:36:06.397658+0000","last_peered":"2024-05-27T12:36:06.397658+0000","last_clean":"2024-05-27T12:36:06.397658+0000","last_became_active":"2024-05-27T12:36:06.393567+0000","last_became_peered":"2024-05-27T12:36:06.393567+0000","last_unstale":"2024-05-27T12:36:06.397658+0000","last_undegraded":"2024-05-27T12:36:06.397658+0000","last_fullsized":"2024-05-27T12:36:06.397658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374467+0000","last_change":"2024-05-27T12:36:06.374467+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374467+0000","last_undegraded":"2024-05-27T12:36:06.374467+0000","last_fullsized":"2024-05-27T12:36:06.374467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402643+0000","last_change":"2024-05-27T12:36:06.396297+0000","last_active":"2024-05-27T12:36:06.402643+0000","last_peered":"2024-05-27T12:36:06.402643+0000","last_clean":"2024-05-27T12:36:06.402643+0000","last_became_active":"2024-05-27T12:36:06.395605+0000","last_became_peered":"2024-05-27T12:36:06.395605+0000","last_unstale":"2024-05-27T12:36:06.402643+0000","last_undegraded":"2024-05-27T12:36:06.402643+0000","last_fullsized":"2024-05-27T12:36:06.402643+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:56:04.614416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373670+0000","last_change":"2024-05-27T12:36:06.373670+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373670+0000","last_undegraded":"2024-05-27T12:36:06.373670+0000","last_fullsized":"2024-05-27T12:36:06.373670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396083+0000","last_change":"2024-05-27T12:36:06.391699+0000","last_active":"2024-05-27T12:36:06.396083+0000","last_peered":"2024-05-27T12:36:06.396083+0000","last_clean":"2024-05-27T12:36:06.396083+0000","last_became_active":"2024-05-27T12:36:06.391285+0000","last_became_peered":"2024-05-27T12:36:06.391285+0000","last_unstale":"2024-05-27T12:36:06.396083+0000","last_undegraded":"2024-05-27T12:36:06.396083+0000","last_fullsized":"2024-05-27T12:36:06.396083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386703+0000","last_change":"2024-05-27T12:36:06.386703+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386703+0000","last_undegraded":"2024-05-27T12:36:06.386703+0000","last_fullsized":"2024-05-27T12:36:06.386703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400607+0000","last_change":"2024-05-27T12:36:06.399046+0000","last_active":"2024-05-27T12:36:06.400607+0000","last_peered":"2024-05-27T12:36:06.400607+0000","last_clean":"2024-05-27T12:36:06.400607+0000","last_became_active":"2024-05-27T12:36:06.396837+0000","last_became_peered":"2024-05-27T12:36:06.396837+0000","last_unstale":"2024-05-27T12:36:06.400607+0000","last_undegraded":"2024-05-27T12:36:06.400607+0000","last_fullsized":"2024-05-27T12:36:06.400607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.376569+0000","last_change":"2024-05-27T12:36:06.376569+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.376569+0000","last_undegraded":"2024-05-27T12:36:06.376569+0000","last_fullsized":"2024-05-27T12:36:06.376569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400216+0000","last_change":"2024-05-27T12:36:06.397002+0000","last_active":"2024-05-27T12:36:06.400216+0000","last_peered":"2024-05-27T12:36:06.400216+0000","last_clean":"2024-05-27T12:36:06.400216+0000","last_became_active":"2024-05-27T12:36:06.396513+0000","last_became_peered":"2024-05-27T12:36:06.396513+0000","last_unstale":"2024-05-27T12:36:06.400216+0000","last_undegraded":"2024-05-27T12:36:06.400216+0000","last_fullsized":"2024-05-27T12:36:06.400216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.382122+0000","last_change":"2024-05-27T12:36:06.382122+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.382122+0000","last_undegraded":"2024-05-27T12:36:06.382122+0000","last_fullsized":"2024-05-27T12:36:06.382122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395817+0000","last_change":"2024-05-27T12:36:06.390945+0000","last_active":"2024-05-27T12:36:06.395817+0000","last_peered":"2024-05-27T12:36:06.395817+0000","last_clean":"2024-05-27T12:36:06.395817+0000","last_became_active":"2024-05-27T12:36:06.390575+0000","last_became_peered":"2024-05-27T12:36:06.390575+0000","last_unstale":"2024-05-27T12:36:06.395817+0000","last_undegraded":"2024-05-27T12:36:06.395817+0000","last_fullsized":"2024-05-27T12:36:06.395817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.384033+0000","last_change":"2024-05-27T12:36:06.384033+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.384033+0000","last_undegraded":"2024-05-27T12:36:06.384033+0000","last_fullsized":"2024-05-27T12:36:06.384033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.402551+0000","last_change":"2024-05-27T12:36:06.396687+0000","last_active":"2024-05-27T12:36:06.402551+0000","last_peered":"2024-05-27T12:36:06.402551+0000","last_clean":"2024-05-27T12:36:06.402551+0000","last_became_active":"2024-05-27T12:36:06.395340+0000","last_became_peered":"2024-05-27T12:36:06.395340+0000","last_unstale":"2024-05-27T12:36:06.402551+0000","last_undegraded":"2024-05-27T12:36:06.402551+0000","last_fullsized":"2024-05-27T12:36:06.402551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:14.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373578+0000","last_change":"2024-05-27T12:36:06.373578+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373578+0000","last_undegraded":"2024-05-27T12:36:06.373578+0000","last_fullsized":"2024-05-27T12:36:06.373578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399923+0000","last_change":"2024-05-27T12:36:06.393791+0000","last_active":"2024-05-27T12:36:06.399923+0000","last_peered":"2024-05-27T12:36:06.399923+0000","last_clean":"2024-05-27T12:36:06.399923+0000","last_became_active":"2024-05-27T12:36:06.391768+0000","last_became_peered":"2024-05-27T12:36:06.391768+0000","last_unstale":"2024-05-27T12:36:06.399923+0000","last_undegraded":"2024-05-27T12:36:06.399923+0000","last_fullsized":"2024-05-27T12:36:06.399923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:11:10.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.380582+0000","last_change":"2024-05-27T12:36:06.380582+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.380582+0000","last_undegraded":"2024-05-27T12:36:06.380582+0000","last_fullsized":"2024-05-27T12:36:06.380582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396504+0000","last_change":"2024-05-27T12:36:06.388667+0000","last_active":"2024-05-27T12:36:06.396504+0000","last_peered":"2024-05-27T12:36:06.396504+0000","last_clean":"2024-05-27T12:36:06.396504+0000","last_became_active":"2024-05-27T12:36:06.388369+0000","last_became_peered":"2024-05-27T12:36:06.388369+0000","last_unstale":"2024-05-27T12:36:06.396504+0000","last_undegraded":"2024-05-27T12:36:06.396504+0000","last_fullsized":"2024-05-27T12:36:06.396504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:49.907565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374535+0000","last_change":"2024-05-27T12:36:06.374535+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374535+0000","last_undegraded":"2024-05-27T12:36:06.374535+0000","last_fullsized":"2024-05-27T12:36:06.374535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394967+0000","last_change":"2024-05-27T12:36:06.391794+0000","last_active":"2024-05-27T12:36:06.394967+0000","last_peered":"2024-05-27T12:36:06.394967+0000","last_clean":"2024-05-27T12:36:06.394967+0000","last_became_active":"2024-05-27T12:36:06.389977+0000","last_became_peered":"2024-05-27T12:36:06.389977+0000","last_unstale":"2024-05-27T12:36:06.394967+0000","last_undegraded":"2024-05-27T12:36:06.394967+0000","last_fullsized":"2024-05-27T12:36:06.394967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373084+0000","last_change":"2024-05-27T12:36:06.373084+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373084+0000","last_undegraded":"2024-05-27T12:36:06.373084+0000","last_fullsized":"2024-05-27T12:36:06.373084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.397674+0000","last_change":"2024-05-27T12:36:06.393310+0000","last_active":"2024-05-27T12:36:06.397674+0000","last_peered":"2024-05-27T12:36:06.397674+0000","last_clean":"2024-05-27T12:36:06.397674+0000","last_became_active":"2024-05-27T12:36:06.392014+0000","last_became_peered":"2024-05-27T12:36:06.392014+0000","last_unstale":"2024-05-27T12:36:06.397674+0000","last_undegraded":"2024-05-27T12:36:06.397674+0000","last_fullsized":"2024-05-27T12:36:06.397674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.374186+0000","last_change":"2024-05-27T12:36:06.374186+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.374186+0000","last_undegraded":"2024-05-27T12:36:06.374186+0000","last_fullsized":"2024-05-27T12:36:06.374186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.399777+0000","last_change":"2024-05-27T12:36:06.393400+0000","last_active":"2024-05-27T12:36:06.399777+0000","last_peered":"2024-05-27T12:36:06.399777+0000","last_clean":"2024-05-27T12:36:06.399777+0000","last_became_active":"2024-05-27T12:36:06.392996+0000","last_became_peered":"2024-05-27T12:36:06.392996+0000","last_unstale":"2024-05-27T12:36:06.399777+0000","last_undegraded":"2024-05-27T12:36:06.399777+0000","last_fullsized":"2024-05-27T12:36:06.399777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:27:30.708117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.386244+0000","last_change":"2024-05-27T12:36:06.386244+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.386244+0000","last_undegraded":"2024-05-27T12:36:06.386244+0000","last_fullsized":"2024-05-27T12:36:06.386244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400560+0000","last_change":"2024-05-27T12:36:06.396022+0000","last_active":"2024-05-27T12:36:06.400560+0000","last_peered":"2024-05-27T12:36:06.400560+0000","last_clean":"2024-05-27T12:36:06.400560+0000","last_became_active":"2024-05-27T12:36:06.395727+0000","last_became_peered":"2024-05-27T12:36:06.395727+0000","last_unstale":"2024-05-27T12:36:06.400560+0000","last_undegraded":"2024-05-27T12:36:06.400560+0000","last_fullsized":"2024-05-27T12:36:06.400560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:03.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379447+0000","last_change":"2024-05-27T12:36:06.379447+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379447+0000","last_undegraded":"2024-05-27T12:36:06.379447+0000","last_fullsized":"2024-05-27T12:36:06.379447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395838+0000","last_change":"2024-05-27T12:36:06.393032+0000","last_active":"2024-05-27T12:36:06.395838+0000","last_peered":"2024-05-27T12:36:06.395838+0000","last_clean":"2024-05-27T12:36:06.395838+0000","last_became_active":"2024-05-27T12:36:06.392754+0000","last_became_peered":"2024-05-27T12:36:06.392754+0000","last_unstale":"2024-05-27T12:36:06.395838+0000","last_undegraded":"2024-05-27T12:36:06.395838+0000","last_fullsized":"2024-05-27T12:36:06.395838+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.379855+0000","last_change":"2024-05-27T12:36:06.379855+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.379855+0000","last_undegraded":"2024-05-27T12:36:06.379855+0000","last_fullsized":"2024-05-27T12:36:06.379855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.401022+0000","last_change":"2024-05-27T12:36:06.399546+0000","last_active":"2024-05-27T12:36:06.401022+0000","last_peered":"2024-05-27T12:36:06.401022+0000","last_clean":"2024-05-27T12:36:06.401022+0000","last_became_active":"2024-05-27T12:36:06.399228+0000","last_became_peered":"2024-05-27T12:36:06.399228+0000","last_unstale":"2024-05-27T12:36:06.401022+0000","last_undegraded":"2024-05-27T12:36:06.401022+0000","last_fullsized":"2024-05-27T12:36:06.401022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375413+0000","last_change":"2024-05-27T12:36:06.375413+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375413+0000","last_undegraded":"2024-05-27T12:36:06.375413+0000","last_fullsized":"2024-05-27T12:36:06.375413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.395586+0000","last_change":"2024-05-27T12:36:06.391239+0000","last_active":"2024-05-27T12:36:06.395586+0000","last_peered":"2024-05-27T12:36:06.395586+0000","last_clean":"2024-05-27T12:36:06.395586+0000","last_became_active":"2024-05-27T12:36:06.390800+0000","last_became_peered":"2024-05-27T12:36:06.390800+0000","last_unstale":"2024-05-27T12:36:06.395586+0000","last_undegraded":"2024-05-27T12:36:06.395586+0000","last_fullsized":"2024-05-27T12:36:06.395586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.375292+0000","last_change":"2024-05-27T12:36:06.375292+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.375292+0000","last_undegraded":"2024-05-27T12:36:06.375292+0000","last_fullsized":"2024-05-27T12:36:06.375292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.391355+0000","last_change":"2024-05-27T12:36:06.389001+0000","last_active":"2024-05-27T12:36:06.391355+0000","last_peered":"2024-05-27T12:36:06.391355+0000","last_clean":"2024-05-27T12:36:06.391355+0000","last_became_active":"2024-05-27T12:36:06.388700+0000","last_became_peered":"2024-05-27T12:36:06.388700+0000","last_unstale":"2024-05-27T12:36:06.391355+0000","last_undegraded":"2024-05-27T12:36:06.391355+0000","last_fullsized":"2024-05-27T12:36:06.391355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.372335+0000","last_change":"2024-05-27T12:36:06.372335+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.372335+0000","last_undegraded":"2024-05-27T12:36:06.372335+0000","last_fullsized":"2024-05-27T12:36:06.372335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.394265+0000","last_change":"2024-05-27T12:36:06.390271+0000","last_active":"2024-05-27T12:36:06.394265+0000","last_peered":"2024-05-27T12:36:06.394265+0000","last_clean":"2024-05-27T12:36:06.394265+0000","last_became_active":"2024-05-27T12:36:06.389933+0000","last_became_peered":"2024-05-27T12:36:06.389933+0000","last_unstale":"2024-05-27T12:36:06.394265+0000","last_undegraded":"2024-05-27T12:36:06.394265+0000","last_fullsized":"2024-05-27T12:36:06.394265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.378119+0000","last_change":"2024-05-27T12:36:06.378119+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.378119+0000","last_undegraded":"2024-05-27T12:36:06.378119+0000","last_fullsized":"2024-05-27T12:36:06.378119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.396869+0000","last_change":"2024-05-27T12:36:06.389847+0000","last_active":"2024-05-27T12:36:06.396869+0000","last_peered":"2024-05-27T12:36:06.396869+0000","last_clean":"2024-05-27T12:36:06.396869+0000","last_became_active":"2024-05-27T12:36:06.389431+0000","last_became_peered":"2024-05-27T12:36:06.389431+0000","last_unstale":"2024-05-27T12:36:06.396869+0000","last_undegraded":"2024-05-27T12:36:06.396869+0000","last_fullsized":"2024-05-27T12:36:06.396869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:54:21.911265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.373815+0000","last_change":"2024-05-27T12:36:06.373815+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.373815+0000","last_undegraded":"2024-05-27T12:36:06.373815+0000","last_fullsized":"2024-05-27T12:36:06.373815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400629+0000","last_change":"2024-05-27T12:36:06.399243+0000","last_active":"2024-05-27T12:36:06.400629+0000","last_peered":"2024-05-27T12:36:06.400629+0000","last_clean":"2024-05-27T12:36:06.400629+0000","last_became_active":"2024-05-27T12:36:06.397252+0000","last_became_peered":"2024-05-27T12:36:06.397252+0000","last_unstale":"2024-05-27T12:36:06.400629+0000","last_undegraded":"2024-05-27T12:36:06.400629+0000","last_fullsized":"2024-05-27T12:36:06.400629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:38.949294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-05-27T12:36:06.383963+0000","last_change":"2024-05-27T12:36:06.383963+0000","last_active":"2024-05-27T12:36:06.346932+0000","last_peered":"2024-05-27T12:36:06.346932+0000","last_clean":"2024-05-27T12:36:06.346932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T12:36:06.383963+0000","last_undegraded":"2024-05-27T12:36:06.383963+0000","last_fullsized":"2024-05-27T12:36:06.383963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.400637+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:06.400637+0000","last_peered":"2024-05-27T12:36:06.400637+0000","last_clean":"2024-05-27T12:36:06.400637+0000","last_became_active":"2024-05-27T12:36:06.398205+0000","last_became_peered":"2024-05-27T12:36:06.398205+0000","last_unstale":"2024-05-27T12:36:06.400637+0000","last_undegraded":"2024-05-27T12:36:06.400637+0000","last_fullsized":"2024-05-27T12:36:06.400637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-27T12:36:06.393078+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:36:06.393078+0000","last_peered":"2024-05-27T12:36:06.393078+0000","last_clean":"2024-05-27T12:36:06.393078+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:36:06.393078+0000","last_undegraded":"2024-05-27T12:36:06.393078+0000","last_fullsized":"2024-05-27T12:36:06.393078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":335872,"data_stored":127274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:36:12.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.380979+0000', 'last_change': '2024-05-27T12:36:06.380979+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.380979+0000', 'last_undegraded': '2024-05-27T12:36:06.380979+0000', 'last_fullsized': '2024-05-27T12:36:06.380979+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381776+0000', 'last_change': '2024-05-27T12:36:06.381776+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381776+0000', 'last_undegraded': '2024-05-27T12:36:06.381776+0000', 'last_fullsized': '2024-05-27T12:36:06.381776+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.379465+0000', 'last_change': '2024-05-27T12:36:06.379465+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.379465+0000', 'last_undegraded': '2024-05-27T12:36:06.379465+0000', 'last_fullsized': '2024-05-27T12:36:06.379465+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374883+0000', 'last_change': '2024-05-27T12:36:06.374883+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374883+0000', 'last_undegraded': '2024-05-27T12:36:06.374883+0000', 'last_fullsized': '2024-05-27T12:36:06.374883+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374479+0000', 'last_change': '2024-05-27T12:36:06.374479+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374479+0000', 'last_undegraded': '2024-05-27T12:36:06.374479+0000', 'last_fullsized': '2024-05-27T12:36:06.374479+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-05-27T12:36:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.377579+0000', 'last_change': '2024-05-27T12:36:06.377579+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.377579+0000', 'last_undegraded': '2024-05-27T12:36:06.377579+0000', 'last_fullsized': '2024-05-27T12:36:06.377579+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.385452+0000', 'last_change': '2024-05-27T12:36:06.385452+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.385452+0000', 'last_undegraded': '2024-05-27T12:36:06.385452+0000', 'last_fullsized': '2024-05-27T12:36:06.385452+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.387969+0000', 'last_change': '2024-05-27T12:36:06.387969+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.387969+0000', 'last_undegraded': '2024-05-27T12:36:06.387969+0000', 'last_fullsized': '2024-05-27T12:36:06.387969+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.382637+0000', 'last_change': '2024-05-27T12:36:06.382637+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.382637+0000', 'last_undegraded': '2024-05-27T12:36:06.382637+0000', 'last_fullsized': '2024-05-27T12:36:06.382637+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.379941+0000', 'last_change': '2024-05-27T12:36:06.379941+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.379941+0000', 'last_undegraded': '2024-05-27T12:36:06.379941+0000', 'last_fullsized': '2024-05-27T12:36:06.379941+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.389103+0000', 'last_change': '2024-05-27T12:36:06.389103+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.389103+0000', 'last_undegraded': '2024-05-27T12:36:06.389103+0000', 'last_fullsized': '2024-05-27T12:36:06.389103+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381444+0000', 'last_change': '2024-05-27T12:36:06.381444+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381444+0000', 'last_undegraded': '2024-05-27T12:36:06.381444+0000', 'last_fullsized': '2024-05-27T12:36:06.381444+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373801+0000', 'last_change': '2024-05-27T12:36:06.373801+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373801+0000', 'last_undegraded': '2024-05-27T12:36:06.373801+0000', 'last_fullsized': '2024-05-27T12:36:06.373801+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376906+0000', 'last_change': '2024-05-27T12:36:06.376906+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376906+0000', 'last_undegraded': '2024-05-27T12:36:06.376906+0000', 'last_fullsized': '2024-05-27T12:36:06.376906+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.377394+0000', 'last_change': '2024-05-27T12:36:06.377394+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.377394+0000', 'last_undegraded': '2024-05-27T12:36:06.377394+0000', 'last_fullsized': '2024-05-27T12:36:06.377394+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.384646+0000', 'last_change': '2024-05-27T12:36:06.384646+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.384646+0000', 'last_undegraded': '2024-05-27T12:36:06.384646+0000', 'last_fullsized': '2024-05-27T12:36:06.384646+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-05-27T12:36:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.391236+0000', 'last_change': '2024-05-27T12:36:06.391236+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.391236+0000', 'last_undegraded': '2024-05-27T12:36:06.391236+0000', 'last_fullsized': '2024-05-27T12:36:06.391236+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381233+0000', 'last_change': '2024-05-27T12:36:06.381233+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381233+0000', 'last_undegraded': '2024-05-27T12:36:06.381233+0000', 'last_fullsized': '2024-05-27T12:36:06.381233+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376290+0000', 'last_change': '2024-05-27T12:36:06.376290+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376290+0000', 'last_undegraded': '2024-05-27T12:36:06.376290+0000', 'last_fullsized': '2024-05-27T12:36:06.376290+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376387+0000', 'last_change': '2024-05-27T12:36:06.376387+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376387+0000', 'last_undegraded': '2024-05-27T12:36:06.376387+0000', 'last_fullsized': '2024-05-27T12:36:06.376387+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381597+0000', 'last_change': '2024-05-27T12:36:06.381597+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381597+0000', 'last_undegraded': '2024-05-27T12:36:06.381597+0000', 'last_fullsized': '2024-05-27T12:36:06.381597+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.384281+0000', 'last_change': '2024-05-27T12:36:06.384281+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.384281+0000', 'last_undegraded': '2024-05-27T12:36:06.384281+0000', 'last_fullsized': '2024-05-27T12:36:06.384281+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373812+0000', 'last_change': '2024-05-27T12:36:06.373812+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373812+0000', 'last_undegraded': '2024-05-27T12:36:06.373812+0000', 'last_fullsized': '2024-05-27T12:36:06.373812+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376671+0000', 'last_change': '2024-05-27T12:36:06.376671+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376671+0000', 'last_undegraded': '2024-05-27T12:36:06.376671+0000', 'last_fullsized': '2024-05-27T12:36:06.376671+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381990+0000', 'last_change': '2024-05-27T12:36:06.381990+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381990+0000', 'last_undegraded': '2024-05-27T12:36:06.381990+0000', 'last_fullsized': '2024-05-27T12:36:06.381990+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.380336+0000', 'last_change': '2024-05-27T12:36:06.380336+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.380336+0000', 'last_undegraded': '2024-05-27T12:36:06.380336+0000', 'last_fullsized': '2024-05-27T12:36:06.380336+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.382812+0000', 'last_change': '2024-05-27T12:36:06.382812+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.382812+0000', 'last_undegraded': '2024-05-27T12:36:06.382812+0000', 'last_fullsized': '2024-05-27T12:36:06.382812+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.380974+0000', 'last_change': '2024-05-27T12:36:06.380974+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.380974+0000', 'last_undegraded': '2024-05-27T12:36:06.380974+0000', 'last_fullsized': '2024-05-27T12:36:06.380974+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.381165+0000', 'last_change': '2024-05-27T12:36:06.381165+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.381165+0000', 'last_undegraded': '2024-05-27T12:36:06.381165+0000', 'last_fullsized': '2024-05-27T12:36:06.381165+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.384415+0000', 'last_change': '2024-05-27T12:36:06.384415+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.384415+0000', 'last_undegraded': '2024-05-27T12:36:06.384415+0000', 'last_fullsized': '2024-05-27T12:36:06.384415+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.378202+0000', 'last_change': '2024-05-27T12:36:06.378202+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.378202+0000', 'last_undegraded': '2024-05-27T12:36:06.378202+0000', 'last_fullsized': '2024-05-27T12:36:06.378202+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.387285+0000', 'last_change': '2024-05-27T12:36:06.387285+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.387285+0000', 'last_undegraded': '2024-05-27T12:36:06.387285+0000', 'last_fullsized': '2024-05-27T12:36:06.387285+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373007+0000', 'last_change': '2024-05-27T12:36:06.373007+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373007+0000', 'last_undegraded': '2024-05-27T12:36:06.373007+0000', 'last_fullsized': '2024-05-27T12:36:06.373007+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-05-27T12:36:12.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.377354+0000', 'last_change': '2024-05-27T12:36:06.377354+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.377354+0000', 'last_undegraded': '2024-05-27T12:36:06.377354+0000', 'last_fullsized': '2024-05-27T12:36:06.377354+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373152+0000', 'last_change': '2024-05-27T12:36:06.373152+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373152+0000', 'last_undegraded': '2024-05-27T12:36:06.373152+0000', 'last_fullsized': '2024-05-27T12:36:06.373152+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.382381+0000', 'last_change': '2024-05-27T12:36:06.382381+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.382381+0000', 'last_undegraded': '2024-05-27T12:36:06.382381+0000', 'last_fullsized': '2024-05-27T12:36:06.382381+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.383806+0000', 'last_change': '2024-05-27T12:36:06.383806+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.383806+0000', 'last_undegraded': '2024-05-27T12:36:06.383806+0000', 'last_fullsized': '2024-05-27T12:36:06.383806+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.382750+0000', 'last_change': '2024-05-27T12:36:06.382750+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.382750+0000', 'last_undegraded': '2024-05-27T12:36:06.382750+0000', 'last_fullsized': '2024-05-27T12:36:06.382750+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376119+0000', 'last_change': '2024-05-27T12:36:06.376119+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376119+0000', 'last_undegraded': '2024-05-27T12:36:06.376119+0000', 'last_fullsized': '2024-05-27T12:36:06.376119+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374894+0000', 'last_change': '2024-05-27T12:36:06.374894+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374894+0000', 'last_undegraded': '2024-05-27T12:36:06.374894+0000', 'last_fullsized': '2024-05-27T12:36:06.374894+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.384618+0000', 'last_change': '2024-05-27T12:36:06.384618+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.384618+0000', 'last_undegraded': '2024-05-27T12:36:06.384618+0000', 'last_fullsized': '2024-05-27T12:36:06.384618+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376474+0000', 'last_change': '2024-05-27T12:36:06.376474+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376474+0000', 'last_undegraded': '2024-05-27T12:36:06.376474+0000', 'last_fullsized': '2024-05-27T12:36:06.376474+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373233+0000', 'last_change': '2024-05-27T12:36:06.373233+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373233+0000', 'last_undegraded': '2024-05-27T12:36:06.373233+0000', 'last_fullsized': '2024-05-27T12:36:06.373233+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.379981+0000', 'last_change': '2024-05-27T12:36:06.379981+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.379981+0000', 'last_undegraded': '2024-05-27T12:36:06.379981+0000', 'last_fullsized': '2024-05-27T12:36:06.379981+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-05-27T12:36:12.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374467+0000', 'last_change': '2024-05-27T12:36:06.374467+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374467+0000', 'last_undegraded': '2024-05-27T12:36:06.374467+0000', 'last_fullsized': '2024-05-27T12:36:06.374467+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373670+0000', 'last_change': '2024-05-27T12:36:06.373670+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373670+0000', 'last_undegraded': '2024-05-27T12:36:06.373670+0000', 'last_fullsized': '2024-05-27T12:36:06.373670+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.386703+0000', 'last_change': '2024-05-27T12:36:06.386703+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.386703+0000', 'last_undegraded': '2024-05-27T12:36:06.386703+0000', 'last_fullsized': '2024-05-27T12:36:06.386703+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.376569+0000', 'last_change': '2024-05-27T12:36:06.376569+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.376569+0000', 'last_undegraded': '2024-05-27T12:36:06.376569+0000', 'last_fullsized': '2024-05-27T12:36:06.376569+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.382122+0000', 'last_change': '2024-05-27T12:36:06.382122+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.382122+0000', 'last_undegraded': '2024-05-27T12:36:06.382122+0000', 'last_fullsized': '2024-05-27T12:36:06.382122+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.384033+0000', 'last_change': '2024-05-27T12:36:06.384033+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.384033+0000', 'last_undegraded': '2024-05-27T12:36:06.384033+0000', 'last_fullsized': '2024-05-27T12:36:06.384033+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-05-27T12:36:12.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373578+0000', 'last_change': '2024-05-27T12:36:06.373578+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373578+0000', 'last_undegraded': '2024-05-27T12:36:06.373578+0000', 'last_fullsized': '2024-05-27T12:36:06.373578+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.380582+0000', 'last_change': '2024-05-27T12:36:06.380582+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.380582+0000', 'last_undegraded': '2024-05-27T12:36:06.380582+0000', 'last_fullsized': '2024-05-27T12:36:06.380582+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374535+0000', 'last_change': '2024-05-27T12:36:06.374535+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374535+0000', 'last_undegraded': '2024-05-27T12:36:06.374535+0000', 'last_fullsized': '2024-05-27T12:36:06.374535+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373084+0000', 'last_change': '2024-05-27T12:36:06.373084+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373084+0000', 'last_undegraded': '2024-05-27T12:36:06.373084+0000', 'last_fullsized': '2024-05-27T12:36:06.373084+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.374186+0000', 'last_change': '2024-05-27T12:36:06.374186+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.374186+0000', 'last_undegraded': '2024-05-27T12:36:06.374186+0000', 'last_fullsized': '2024-05-27T12:36:06.374186+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.386244+0000', 'last_change': '2024-05-27T12:36:06.386244+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.386244+0000', 'last_undegraded': '2024-05-27T12:36:06.386244+0000', 'last_fullsized': '2024-05-27T12:36:06.386244+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.379447+0000', 'last_change': '2024-05-27T12:36:06.379447+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.379447+0000', 'last_undegraded': '2024-05-27T12:36:06.379447+0000', 'last_fullsized': '2024-05-27T12:36:06.379447+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.379855+0000', 'last_change': '2024-05-27T12:36:06.379855+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.379855+0000', 'last_undegraded': '2024-05-27T12:36:06.379855+0000', 'last_fullsized': '2024-05-27T12:36:06.379855+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.375413+0000', 'last_change': '2024-05-27T12:36:06.375413+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.375413+0000', 'last_undegraded': '2024-05-27T12:36:06.375413+0000', 'last_fullsized': '2024-05-27T12:36:06.375413+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.375292+0000', 'last_change': '2024-05-27T12:36:06.375292+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.375292+0000', 'last_undegraded': '2024-05-27T12:36:06.375292+0000', 'last_fullsized': '2024-05-27T12:36:06.375292+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.372335+0000', 'last_change': '2024-05-27T12:36:06.372335+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.372335+0000', 'last_undegraded': '2024-05-27T12:36:06.372335+0000', 'last_fullsized': '2024-05-27T12:36:06.372335+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-05-27T12:36:12.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.378119+0000', 'last_change': '2024-05-27T12:36:06.378119+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.378119+0000', 'last_undegraded': '2024-05-27T12:36:06.378119+0000', 'last_fullsized': '2024-05-27T12:36:06.378119+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-27T12:36:12.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-05-27T12:36:12.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.373815+0000', 'last_change': '2024-05-27T12:36:06.373815+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.373815+0000', 'last_undegraded': '2024-05-27T12:36:06.373815+0000', 'last_fullsized': '2024-05-27T12:36:06.373815+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-27T12:36:12.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-05-27T12:36:12.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-05-27T12:36:06.383963+0000', 'last_change': '2024-05-27T12:36:06.383963+0000', 'last_active': '2024-05-27T12:36:06.346932+0000', 'last_peered': '2024-05-27T12:36:06.346932+0000', 'last_clean': '2024-05-27T12:36:06.346932+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-27T12:36:06.383963+0000', 'last_undegraded': '2024-05-27T12:36:06.383963+0000', 'last_fullsized': '2024-05-27T12:36:06.383963+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'last_clean_scrub_stamp': '2024-05-27T12:36:06.346932+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-27T12:36:12.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-27T12:36:13.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.067+0000 7fb5ab400640 1 Processor -- start 2024-05-27T12:36:13.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.067+0000 7fb5ab400640 1 -- start start 2024-05-27T12:36:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb5a40ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5ab400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a406dfc0 con 0x7fb5a4072fb0 2024-05-27T12:36:13.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5ab400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a406ed10 con 0x7fb5a4149160 2024-05-27T12:36:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5ab400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a4149540 con 0x7fb5a4072630 2024-05-27T12:36:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5a3e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb5a4072fb0 0x7fb5a4166910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36592/0 (socket says 172.21.15.94:36592) 2024-05-27T12:36:13.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5a3e00640 1 -- 172.21.15.94:0/3720023445 learned_addr learned my addr 172.21.15.94:0/3720023445 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.069+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665229210 0 0) 0x7fb5a406dfc0 con 0x7fb5a4072fb0 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb588003520 con 0x7fb5a4072fb0 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a9400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb5a40ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248084430 0 0) 0x7fb588003520 con 0x7fb5a4072fb0 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 msgr2=0x7fb5a40ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb5a40ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 >> v1:172.21.15.97:6789/0 conn(0x7fb5a4149160 legacy=0x7fb5a4071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a406e230 con 0x7fb5a4072fb0 2024-05-27T12:36:13.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fb590001670 con 0x7fb5a4072fb0 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 >> v1:172.21.15.94:6789/0 conn(0x7fb5a4072fb0 legacy=0x7fb5a4166910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb588003ac0 0x7fb588003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb588004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb588005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4149540 con 0x7fb588004e10 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb58800bc50 con 0x7fb588003ac0 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb58800be50 con 0x7fb5a4072630 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.070+0000 7fb5a9400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb588005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a8a00640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb588003ac0 0x7fb588003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a3e00640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb588004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a9400640 1 -- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 msgr2=0x7fb588004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a9400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a4072630 0x7fb588004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a9400640 1 -- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb588003ac0 msgr2=0x7fb588003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a9400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb588003ac0 0x7fb588003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.071+0000 7fb5a9400640 1 -- 172.21.15.94:0/3720023445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb588003520 con 0x7fb588004e10 2024-05-27T12:36:13.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.072+0000 7fb5a9400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb588005290 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00ba30 tx=0x7fb588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8022e449e4e9e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.072+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c030070 con 0x7fb588004e10 2024-05-27T12:36:13.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.072+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c02ade0 con 0x7fb588004e10 2024-05-27T12:36:13.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.072+0000 7fb5a3400640 1 -- 172.21.15.94:0/3720023445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00b600 con 0x7fb588004e10 2024-05-27T12:36:13.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 -- 172.21.15.94:0/3720023445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 msgr2=0x7fb588005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3720023445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb588005290 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00ba30 tx=0x7fb588005830 comp rx=0 tx=0).stop 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5a9400640 1 -- 172.21.15.94:0/3720023445 reap_dead start 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 -- 172.21.15.94:0/3720023445 shutdown_connections 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 -- 172.21.15.94:0/3720023445 >> 172.21.15.94:0/3720023445 conn(0x7fb5a40b3830 msgr2=0x7fb5a40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 -- 172.21.15.94:0/3720023445 shutdown_connections 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.073+0000 7fb5ab400640 1 -- 172.21.15.94:0/3720023445 wait complete. 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.075+0000 7fb5ab400640 1 Processor -- start 2024-05-27T12:36:13.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.075+0000 7fb5ab400640 1 -- start start 2024-05-27T12:36:13.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a413cf30 0x7fb5a4143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a413f100 0x7fb5a413f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a40cff30 con 0x7fb588004e10 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a40d03a0 con 0x7fb5a413f100 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5ab400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a40d00b0 con 0x7fb5a413cf30 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46274/0 (socket says 172.21.15.94:46274) 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 -- 172.21.15.94:0/3050809605 learned_addr learned my addr 172.21.15.94:0/3050809605 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 -- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a413cf30 msgr2=0x7fb5a4143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a413cf30 0x7fb5a4143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 -- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a413f100 msgr2=0x7fb5a413f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a413f100 0x7fb5a413f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.076+0000 7fb5a8a00640 1 -- 172.21.15.94:0/3050809605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a413d3b0 con 0x7fb588004e10 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5a8a00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e840 tx=0x7fb59400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594003b80 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb594018070 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a4164c80 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940050d0 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.077+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41646c0 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.078+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a4075460 con 0x7fb588004e10 2024-05-27T12:36:13.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.079+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb594005270 con 0x7fb588004e10 2024-05-27T12:36:13.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.080+0000 7fb5a0c00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.080+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb59408ba90 con 0x7fb588004e10 2024-05-27T12:36:13.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.081+0000 7fb5a3e00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.082+0000 7fb5a3e00640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 0x7fb58406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb590001050 tx=0x7fb590000f60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.085+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb594059eb0 con 0x7fb588004e10 2024-05-27T12:36:13.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.336+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb5a40b4100 con 0x7fb588004e10 2024-05-27T12:36:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.337+0000 7fb5a0c00640 1 -- 172.21.15.94:0/3050809605 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1194 (secure 0 0 0) 0x7fb5940597b0 con 0x7fb588004e10 2024-05-27T12:36:13.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":93,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1716813291,"num_in_osds":8,"osd_in_since":1716813283,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242352128,"bytes_avail":772851761152,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4971}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-27T12:35:54.879007+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-27T12:36:13.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 msgr2=0x7fb58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 0x7fb58406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb590001050 tx=0x7fb590000f60 comp rx=0 tx=0).stop 2024-05-27T12:36:13.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 msgr2=0x7fb5a4163d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e840 tx=0x7fb59400ed00 comp rx=0 tx=0).stop 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 shutdown_connections 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5a413f100 0x7fb5a413f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5a413cf30 0x7fb5a4143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 --2- 172.21.15.94:0/3050809605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb588004e10 0x7fb5a4163d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 >> 172.21.15.94:0/3050809605 conn(0x7fb5a40b3830 msgr2=0x7fb5a40cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 shutdown_connections 2024-05-27T12:36:13.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.340+0000 7fb5ab400640 1 -- 172.21.15.94:0/3050809605 wait complete. 2024-05-27T12:36:13.358 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-27T12:36:13.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:36:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.549+0000 7fa77be00640 1 Processor -- start 2024-05-27T12:36:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.549+0000 7fa77be00640 1 -- start start 2024-05-27T12:36:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa77c0d4ae0 0x7fa77c0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77be00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c15ee10 con 0x7fa77c15ea30 2024-05-27T12:36:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c0d2a60 con 0x7fa77c0b4510 2024-05-27T12:36:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0d2830 con 0x7fa77c0d4ae0 2024-05-27T12:36:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77aa00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa77c15ea30 0x7fa77c0d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36596/0 (socket says 172.21.15.94:36596) 2024-05-27T12:36:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77aa00640 1 -- 172.21.15.94:0/198177841 learned_addr learned my addr 172.21.15.94:0/198177841 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa77a000640 1 --2- 172.21.15.94:0/198177841 >> v2:172.21.15.97:3300/0 conn(0x7fa77c0d4ae0 0x7fa77c0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.551+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580109651 0 0) 0x7fa77c15ee10 con 0x7fa77c15ea30 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa760003520 con 0x7fa77c15ea30 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2527626222 0 0) 0x7fa760003520 con 0x7fa77c15ea30 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 >> v2:172.21.15.97:3300/0 conn(0x7fa77c0d4ae0 msgr2=0x7fa77c0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 --2- 172.21.15.94:0/198177841 >> v2:172.21.15.97:3300/0 conn(0x7fa77c0d4ae0 0x7fa77c0d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 >> v1:172.21.15.97:6789/0 conn(0x7fa77c0b4510 legacy=0x7fa77c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0d2d40 con 0x7fa77c15ea30 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fa76c0019f0 con 0x7fa77c15ea30 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 >> v1:172.21.15.94:6789/0 conn(0x7fa77c15ea30 legacy=0x7fa77c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa760003a00 0x7fa760003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa76000c070 con 0x7fa7600051c0 2024-05-27T12:36:13.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa76000c220 con 0x7fa760004800 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa76000c420 con 0x7fa760003a00 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.552+0000 7fa77b400640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77a000640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77aa00640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa760003a00 0x7fa760003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77b400640 1 -- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa760003a00 msgr2=0x7fa760003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77b400640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa760003a00 0x7fa760003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77b400640 1 -- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa760004800 msgr2=0x7fa760004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77b400640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa760004800 0x7fa760004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.553+0000 7fa77b400640 1 -- 172.21.15.94:0/198177841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa760003520 con 0x7fa7600051c0 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77b400640 1 -- 172.21.15.94:0/198177841 reap_dead start 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77b400640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa76000b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa774001750 tx=0x7fa760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd3ff6dd0fe859d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774001370 con 0x7fa7600051c0 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa774005310 con 0x7fa7600051c0 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa779600640 1 -- 172.21.15.94:0/198177841 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77402abe0 con 0x7fa7600051c0 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77be00640 1 -- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 msgr2=0x7fa76000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77be00640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa76000b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa774001750 tx=0x7fa760005830 comp rx=0 tx=0).stop 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77be00640 1 -- 172.21.15.94:0/198177841 shutdown_connections 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77be00640 1 --2- 172.21.15.94:0/198177841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa76000b9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.554+0000 7fa77be00640 1 -- 172.21.15.94:0/198177841 >> 172.21.15.94:0/198177841 conn(0x7fa77c09da70 msgr2=0x7fa77c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.555+0000 7fa77be00640 1 -- 172.21.15.94:0/198177841 shutdown_connections 2024-05-27T12:36:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.555+0000 7fa77be00640 1 -- 172.21.15.94:0/198177841 wait complete. 2024-05-27T12:36:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.556+0000 7fa77be00640 1 Processor -- start 2024-05-27T12:36:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.556+0000 7fa77be00640 1 -- start start 2024-05-27T12:36:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 0x7fa77c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 0x7fa77c0b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c0d35e0 con 0x7fa7600051c0 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c0d3300 con 0x7fa77c153fa0 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.557+0000 7fa77be00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c0d22f0 con 0x7fa77c153b40 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46288/0 (socket says 172.21.15.94:46288) 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 -- 172.21.15.94:0/2338743959 learned_addr learned my addr 172.21.15.94:0/2338743959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77b400640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 0x7fa77c0b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77a000640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 0x7fa77c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 -- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 msgr2=0x7fa77c0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 0x7fa77c0b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 -- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 msgr2=0x7fa77c0b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 0x7fa77c0b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77aa00640 1 -- 172.21.15.94:0/2338743959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c0c9e30 con 0x7fa7600051c0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77a000640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 0x7fa77c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.558+0000 7fa77b400640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 0x7fa77c0b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa77aa00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa76c002070 tx=0x7fa76c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c019070 con 0x7fa7600051c0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c004d60 con 0x7fa7600051c0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0072d0 con 0x7fa7600051c0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c0ca110 con 0x7fa7600051c0 2024-05-27T12:36:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.559+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c0ca6d0 con 0x7fa7600051c0 2024-05-27T12:36:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.560+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c075460 con 0x7fa7600051c0 2024-05-27T12:36:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.561+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa76c007470 con 0x7fa7600051c0 2024-05-27T12:36:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.562+0000 7fa773400640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 0x7fa75c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.562+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa76c08bd50 con 0x7fa7600051c0 2024-05-27T12:36:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.562+0000 7fa77a000640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 0x7fa75c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.563+0000 7fa77a000640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 0x7fa75c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0cdf30 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.566+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa76c05a0e0 con 0x7fa7600051c0 2024-05-27T12:36:13.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.762+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa77c0cfe50 con 0x7fa75c06c8f0 2024-05-27T12:36:13.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.781+0000 7fa773400640 1 -- 172.21.15.94:0/2338743959 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fa77c0cfe50 con 0x7fa75c06c8f0 2024-05-27T12:36:13.782 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:13.785 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 msgr2=0x7fa75c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 0x7fa75c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0cdf30 tx=0x7fa768008040 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 msgr2=0x7fa77c0c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa76c002070 tx=0x7fa76c00b4a0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 shutdown_connections 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa75c06c8f0 0x7fa75c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77c153fa0 0x7fa77c0b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa77c153b40 0x7fa77c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 --2- 172.21.15.94:0/2338743959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7600051c0 0x7fa77c0c9140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 >> 172.21.15.94:0/2338743959 conn(0x7fa77c09da70 msgr2=0x7fa77c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 shutdown_connections 2024-05-27T12:36:13.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:13.787+0000 7fa77be00640 1 -- 172.21.15.94:0/2338743959 wait complete. 2024-05-27T12:36:13.806 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-05-27T12:36:12.884332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236952,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737768,"statfs":{"total":773094113280,"available":772851474432,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.520161"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410969+0000","last_change":"2024-05-27T12:36:07.410152+0000","last_active":"2024-05-27T12:36:08.410969+0000","last_peered":"2024-05-27T12:36:08.410969+0000","last_clean":"2024-05-27T12:36:08.410969+0000","last_became_active":"2024-05-27T12:36:07.407363+0000","last_became_peered":"2024-05-27T12:36:07.407363+0000","last_unstale":"2024-05-27T12:36:08.410969+0000","last_undegraded":"2024-05-27T12:36:08.410969+0000","last_fullsized":"2024-05-27T12:36:08.410969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:45:47.782240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501165+0000","last_change":"2024-05-27T12:36:07.400013+0000","last_active":"2024-05-27T12:36:08.501165+0000","last_peered":"2024-05-27T12:36:08.501165+0000","last_clean":"2024-05-27T12:36:08.501165+0000","last_became_active":"2024-05-27T12:36:07.398865+0000","last_became_peered":"2024-05-27T12:36:07.398865+0000","last_unstale":"2024-05-27T12:36:08.501165+0000","last_undegraded":"2024-05-27T12:36:08.501165+0000","last_fullsized":"2024-05-27T12:36:08.501165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504067+0000","last_change":"2024-05-27T12:36:07.405997+0000","last_active":"2024-05-27T12:36:08.504067+0000","last_peered":"2024-05-27T12:36:08.504067+0000","last_clean":"2024-05-27T12:36:08.504067+0000","last_became_active":"2024-05-27T12:36:07.405695+0000","last_became_peered":"2024-05-27T12:36:07.405695+0000","last_unstale":"2024-05-27T12:36:08.504067+0000","last_undegraded":"2024-05-27T12:36:08.504067+0000","last_fullsized":"2024-05-27T12:36:08.504067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:49:21.153367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421039+0000","last_change":"2024-05-27T12:36:07.419031+0000","last_active":"2024-05-27T12:36:08.421039+0000","last_peered":"2024-05-27T12:36:08.421039+0000","last_clean":"2024-05-27T12:36:08.421039+0000","last_became_active":"2024-05-27T12:36:07.415654+0000","last_became_peered":"2024-05-27T12:36:07.415654+0000","last_unstale":"2024-05-27T12:36:08.421039+0000","last_undegraded":"2024-05-27T12:36:08.421039+0000","last_fullsized":"2024-05-27T12:36:08.421039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:13.677115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500345+0000","last_change":"2024-05-27T12:36:07.398198+0000","last_active":"2024-05-27T12:36:08.500345+0000","last_peered":"2024-05-27T12:36:08.500345+0000","last_clean":"2024-05-27T12:36:08.500345+0000","last_became_active":"2024-05-27T12:36:07.397900+0000","last_became_peered":"2024-05-27T12:36:07.397900+0000","last_unstale":"2024-05-27T12:36:08.500345+0000","last_undegraded":"2024-05-27T12:36:08.500345+0000","last_fullsized":"2024-05-27T12:36:08.500345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:24.314862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500556+0000","last_change":"2024-05-27T12:36:07.406235+0000","last_active":"2024-05-27T12:36:08.500556+0000","last_peered":"2024-05-27T12:36:08.500556+0000","last_clean":"2024-05-27T12:36:08.500556+0000","last_became_active":"2024-05-27T12:36:07.405940+0000","last_became_peered":"2024-05-27T12:36:07.405940+0000","last_unstale":"2024-05-27T12:36:08.500556+0000","last_undegraded":"2024-05-27T12:36:08.500556+0000","last_fullsized":"2024-05-27T12:36:08.500556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:56:42.538427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457814+0000","last_change":"2024-05-27T12:36:07.409550+0000","last_active":"2024-05-27T12:36:08.457814+0000","last_peered":"2024-05-27T12:36:08.457814+0000","last_clean":"2024-05-27T12:36:08.457814+0000","last_became_active":"2024-05-27T12:36:07.408848+0000","last_became_peered":"2024-05-27T12:36:07.408848+0000","last_unstale":"2024-05-27T12:36:08.457814+0000","last_undegraded":"2024-05-27T12:36:08.457814+0000","last_fullsized":"2024-05-27T12:36:08.457814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457537+0000","last_change":"2024-05-27T12:36:07.404370+0000","last_active":"2024-05-27T12:36:08.457537+0000","last_peered":"2024-05-27T12:36:08.457537+0000","last_clean":"2024-05-27T12:36:08.457537+0000","last_became_active":"2024-05-27T12:36:07.403997+0000","last_became_peered":"2024-05-27T12:36:07.403997+0000","last_unstale":"2024-05-27T12:36:08.457537+0000","last_undegraded":"2024-05-27T12:36:08.457537+0000","last_fullsized":"2024-05-27T12:36:08.457537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.414120+0000","last_change":"2024-05-27T12:36:07.401707+0000","last_active":"2024-05-27T12:36:08.414120+0000","last_peered":"2024-05-27T12:36:08.414120+0000","last_clean":"2024-05-27T12:36:08.414120+0000","last_became_active":"2024-05-27T12:36:07.401164+0000","last_became_peered":"2024-05-27T12:36:07.401164+0000","last_unstale":"2024-05-27T12:36:08.414120+0000","last_undegraded":"2024-05-27T12:36:08.414120+0000","last_fullsized":"2024-05-27T12:36:08.414120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:49.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421210+0000","last_change":"2024-05-27T12:36:07.412627+0000","last_active":"2024-05-27T12:36:08.421210+0000","last_peered":"2024-05-27T12:36:08.421210+0000","last_clean":"2024-05-27T12:36:08.421210+0000","last_became_active":"2024-05-27T12:36:07.412376+0000","last_became_peered":"2024-05-27T12:36:07.412376+0000","last_unstale":"2024-05-27T12:36:08.421210+0000","last_undegraded":"2024-05-27T12:36:08.421210+0000","last_fullsized":"2024-05-27T12:36:08.421210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:49.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458107+0000","last_change":"2024-05-27T12:36:07.405204+0000","last_active":"2024-05-27T12:36:08.458107+0000","last_peered":"2024-05-27T12:36:08.458107+0000","last_clean":"2024-05-27T12:36:08.458107+0000","last_became_active":"2024-05-27T12:36:07.404860+0000","last_became_peered":"2024-05-27T12:36:07.404860+0000","last_unstale":"2024-05-27T12:36:08.458107+0000","last_undegraded":"2024-05-27T12:36:08.458107+0000","last_fullsized":"2024-05-27T12:36:08.458107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503523+0000","last_change":"2024-05-27T12:36:07.406970+0000","last_active":"2024-05-27T12:36:08.503523+0000","last_peered":"2024-05-27T12:36:08.503523+0000","last_clean":"2024-05-27T12:36:08.503523+0000","last_became_active":"2024-05-27T12:36:07.406196+0000","last_became_peered":"2024-05-27T12:36:07.406196+0000","last_unstale":"2024-05-27T12:36:08.503523+0000","last_undegraded":"2024-05-27T12:36:08.503523+0000","last_fullsized":"2024-05-27T12:36:08.503523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500506+0000","last_change":"2024-05-27T12:36:07.412844+0000","last_active":"2024-05-27T12:36:08.500506+0000","last_peered":"2024-05-27T12:36:08.500506+0000","last_clean":"2024-05-27T12:36:08.500506+0000","last_became_active":"2024-05-27T12:36:07.412220+0000","last_became_peered":"2024-05-27T12:36:07.412220+0000","last_unstale":"2024-05-27T12:36:08.500506+0000","last_undegraded":"2024-05-27T12:36:08.500506+0000","last_fullsized":"2024-05-27T12:36:08.500506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:29.723914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500598+0000","last_change":"2024-05-27T12:36:07.401572+0000","last_active":"2024-05-27T12:36:08.500598+0000","last_peered":"2024-05-27T12:36:08.500598+0000","last_clean":"2024-05-27T12:36:08.500598+0000","last_became_active":"2024-05-27T12:36:07.401201+0000","last_became_peered":"2024-05-27T12:36:07.401201+0000","last_unstale":"2024-05-27T12:36:08.500598+0000","last_undegraded":"2024-05-27T12:36:08.500598+0000","last_fullsized":"2024-05-27T12:36:08.500598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:22:03.210565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503410+0000","last_change":"2024-05-27T12:36:07.407396+0000","last_active":"2024-05-27T12:36:08.503410+0000","last_peered":"2024-05-27T12:36:08.503410+0000","last_clean":"2024-05-27T12:36:08.503410+0000","last_became_active":"2024-05-27T12:36:07.407082+0000","last_became_peered":"2024-05-27T12:36:07.407082+0000","last_unstale":"2024-05-27T12:36:08.503410+0000","last_undegraded":"2024-05-27T12:36:08.503410+0000","last_fullsized":"2024-05-27T12:36:08.503410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458466+0000","last_change":"2024-05-27T12:36:07.403712+0000","last_active":"2024-05-27T12:36:08.458466+0000","last_peered":"2024-05-27T12:36:08.458466+0000","last_clean":"2024-05-27T12:36:08.458466+0000","last_became_active":"2024-05-27T12:36:07.402513+0000","last_became_peered":"2024-05-27T12:36:07.402513+0000","last_unstale":"2024-05-27T12:36:08.458466+0000","last_undegraded":"2024-05-27T12:36:08.458466+0000","last_fullsized":"2024-05-27T12:36:08.458466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:47:21.373015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458387+0000","last_change":"2024-05-27T12:36:07.408098+0000","last_active":"2024-05-27T12:36:08.458387+0000","last_peered":"2024-05-27T12:36:08.458387+0000","last_clean":"2024-05-27T12:36:08.458387+0000","last_became_active":"2024-05-27T12:36:07.406544+0000","last_became_peered":"2024-05-27T12:36:07.406544+0000","last_unstale":"2024-05-27T12:36:08.458387+0000","last_undegraded":"2024-05-27T12:36:08.458387+0000","last_fullsized":"2024-05-27T12:36:08.458387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:24.704060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501215+0000","last_change":"2024-05-27T12:36:07.409467+0000","last_active":"2024-05-27T12:36:08.501215+0000","last_peered":"2024-05-27T12:36:08.501215+0000","last_clean":"2024-05-27T12:36:08.501215+0000","last_became_active":"2024-05-27T12:36:07.409210+0000","last_became_peered":"2024-05-27T12:36:07.409210+0000","last_unstale":"2024-05-27T12:36:08.501215+0000","last_undegraded":"2024-05-27T12:36:08.501215+0000","last_fullsized":"2024-05-27T12:36:08.501215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500944+0000","last_change":"2024-05-27T12:36:07.402927+0000","last_active":"2024-05-27T12:36:08.500944+0000","last_peered":"2024-05-27T12:36:08.500944+0000","last_clean":"2024-05-27T12:36:08.500944+0000","last_became_active":"2024-05-27T12:36:07.402642+0000","last_became_peered":"2024-05-27T12:36:07.402642+0000","last_unstale":"2024-05-27T12:36:08.500944+0000","last_undegraded":"2024-05-27T12:36:08.500944+0000","last_fullsized":"2024-05-27T12:36:08.500944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500502+0000","last_change":"2024-05-27T12:36:07.408032+0000","last_active":"2024-05-27T12:36:08.500502+0000","last_peered":"2024-05-27T12:36:08.500502+0000","last_clean":"2024-05-27T12:36:08.500502+0000","last_became_active":"2024-05-27T12:36:07.407755+0000","last_became_peered":"2024-05-27T12:36:07.407755+0000","last_unstale":"2024-05-27T12:36:08.500502+0000","last_undegraded":"2024-05-27T12:36:08.500502+0000","last_fullsized":"2024-05-27T12:36:08.500502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:05.666798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411283+0000","last_change":"2024-05-27T12:36:07.399185+0000","last_active":"2024-05-27T12:36:08.411283+0000","last_peered":"2024-05-27T12:36:08.411283+0000","last_clean":"2024-05-27T12:36:08.411283+0000","last_became_active":"2024-05-27T12:36:07.398851+0000","last_became_peered":"2024-05-27T12:36:07.398851+0000","last_unstale":"2024-05-27T12:36:08.411283+0000","last_undegraded":"2024-05-27T12:36:08.411283+0000","last_fullsized":"2024-05-27T12:36:08.411283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:16.556947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410524+0000","last_change":"2024-05-27T12:36:07.394814+0000","last_active":"2024-05-27T12:36:08.410524+0000","last_peered":"2024-05-27T12:36:08.410524+0000","last_clean":"2024-05-27T12:36:08.410524+0000","last_became_active":"2024-05-27T12:36:07.394471+0000","last_became_peered":"2024-05-27T12:36:07.394471+0000","last_unstale":"2024-05-27T12:36:08.410524+0000","last_undegraded":"2024-05-27T12:36:08.410524+0000","last_fullsized":"2024-05-27T12:36:08.410524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:41.784350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406885+0000","last_change":"2024-05-27T12:36:07.397769+0000","last_active":"2024-05-27T12:36:08.406885+0000","last_peered":"2024-05-27T12:36:08.406885+0000","last_clean":"2024-05-27T12:36:08.406885+0000","last_became_active":"2024-05-27T12:36:07.397423+0000","last_became_peered":"2024-05-27T12:36:07.397423+0000","last_unstale":"2024-05-27T12:36:08.406885+0000","last_undegraded":"2024-05-27T12:36:08.406885+0000","last_fullsized":"2024-05-27T12:36:08.406885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:24.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503104+0000","last_change":"2024-05-27T12:36:07.400408+0000","last_active":"2024-05-27T12:36:08.503104+0000","last_peered":"2024-05-27T12:36:08.503104+0000","last_clean":"2024-05-27T12:36:08.503104+0000","last_became_active":"2024-05-27T12:36:07.400021+0000","last_became_peered":"2024-05-27T12:36:07.400021+0000","last_unstale":"2024-05-27T12:36:08.503104+0000","last_undegraded":"2024-05-27T12:36:08.503104+0000","last_fullsized":"2024-05-27T12:36:08.503104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421380+0000","last_change":"2024-05-27T12:36:07.404454+0000","last_active":"2024-05-27T12:36:08.421380+0000","last_peered":"2024-05-27T12:36:08.421380+0000","last_clean":"2024-05-27T12:36:08.421380+0000","last_became_active":"2024-05-27T12:36:07.403996+0000","last_became_peered":"2024-05-27T12:36:07.403996+0000","last_unstale":"2024-05-27T12:36:08.421380+0000","last_undegraded":"2024-05-27T12:36:08.421380+0000","last_fullsized":"2024-05-27T12:36:08.421380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:01.530801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501313+0000","last_change":"2024-05-27T12:36:07.398861+0000","last_active":"2024-05-27T12:36:08.501313+0000","last_peered":"2024-05-27T12:36:08.501313+0000","last_clean":"2024-05-27T12:36:08.501313+0000","last_became_active":"2024-05-27T12:36:07.398499+0000","last_became_peered":"2024-05-27T12:36:07.398499+0000","last_unstale":"2024-05-27T12:36:08.501313+0000","last_undegraded":"2024-05-27T12:36:08.501313+0000","last_fullsized":"2024-05-27T12:36:08.501313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421309+0000","last_change":"2024-05-27T12:36:07.412138+0000","last_active":"2024-05-27T12:36:08.421309+0000","last_peered":"2024-05-27T12:36:08.421309+0000","last_clean":"2024-05-27T12:36:08.421309+0000","last_became_active":"2024-05-27T12:36:07.411322+0000","last_became_peered":"2024-05-27T12:36:07.411322+0000","last_unstale":"2024-05-27T12:36:08.421309+0000","last_undegraded":"2024-05-27T12:36:08.421309+0000","last_fullsized":"2024-05-27T12:36:08.421309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:32:57.241855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503629+0000","last_change":"2024-05-27T12:36:07.404147+0000","last_active":"2024-05-27T12:36:08.503629+0000","last_peered":"2024-05-27T12:36:08.503629+0000","last_clean":"2024-05-27T12:36:08.503629+0000","last_became_active":"2024-05-27T12:36:07.403456+0000","last_became_peered":"2024-05-27T12:36:07.403456+0000","last_unstale":"2024-05-27T12:36:08.503629+0000","last_undegraded":"2024-05-27T12:36:08.503629+0000","last_fullsized":"2024-05-27T12:36:08.503629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411653+0000","last_change":"2024-05-27T12:36:07.397416+0000","last_active":"2024-05-27T12:36:08.411653+0000","last_peered":"2024-05-27T12:36:08.411653+0000","last_clean":"2024-05-27T12:36:08.411653+0000","last_became_active":"2024-05-27T12:36:07.394550+0000","last_became_peered":"2024-05-27T12:36:07.394550+0000","last_unstale":"2024-05-27T12:36:08.411653+0000","last_undegraded":"2024-05-27T12:36:08.411653+0000","last_fullsized":"2024-05-27T12:36:08.411653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:48:07.303685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457670+0000","last_change":"2024-05-27T12:36:07.406182+0000","last_active":"2024-05-27T12:36:08.457670+0000","last_peered":"2024-05-27T12:36:08.457670+0000","last_clean":"2024-05-27T12:36:08.457670+0000","last_became_active":"2024-05-27T12:36:07.405895+0000","last_became_peered":"2024-05-27T12:36:07.405895+0000","last_unstale":"2024-05-27T12:36:08.457670+0000","last_undegraded":"2024-05-27T12:36:08.457670+0000","last_fullsized":"2024-05-27T12:36:08.457670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411216+0000","last_change":"2024-05-27T12:36:07.400912+0000","last_active":"2024-05-27T12:36:08.411216+0000","last_peered":"2024-05-27T12:36:08.411216+0000","last_clean":"2024-05-27T12:36:08.411216+0000","last_became_active":"2024-05-27T12:36:07.400267+0000","last_became_peered":"2024-05-27T12:36:07.400267+0000","last_unstale":"2024-05-27T12:36:08.411216+0000","last_undegraded":"2024-05-27T12:36:08.411216+0000","last_fullsized":"2024-05-27T12:36:08.411216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:33:12.319943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457720+0000","last_change":"2024-05-27T12:36:07.407534+0000","last_active":"2024-05-27T12:36:08.457720+0000","last_peered":"2024-05-27T12:36:08.457720+0000","last_clean":"2024-05-27T12:36:08.457720+0000","last_became_active":"2024-05-27T12:36:07.406424+0000","last_became_peered":"2024-05-27T12:36:07.406424+0000","last_unstale":"2024-05-27T12:36:08.457720+0000","last_undegraded":"2024-05-27T12:36:08.457720+0000","last_fullsized":"2024-05-27T12:36:08.457720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406923+0000","last_change":"2024-05-27T12:36:07.400046+0000","last_active":"2024-05-27T12:36:08.406923+0000","last_peered":"2024-05-27T12:36:08.406923+0000","last_clean":"2024-05-27T12:36:08.406923+0000","last_became_active":"2024-05-27T12:36:07.399770+0000","last_became_peered":"2024-05-27T12:36:07.399770+0000","last_unstale":"2024-05-27T12:36:08.406923+0000","last_undegraded":"2024-05-27T12:36:08.406923+0000","last_fullsized":"2024-05-27T12:36:08.406923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:41.784350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411657+0000","last_change":"2024-05-27T12:36:06.399603+0000","last_active":"2024-05-27T12:36:08.411657+0000","last_peered":"2024-05-27T12:36:08.411657+0000","last_clean":"2024-05-27T12:36:08.411657+0000","last_became_active":"2024-05-27T12:36:06.399286+0000","last_became_peered":"2024-05-27T12:36:06.399286+0000","last_unstale":"2024-05-27T12:36:08.411657+0000","last_undegraded":"2024-05-27T12:36:08.411657+0000","last_fullsized":"2024-05-27T12:36:08.411657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411395+0000","last_change":"2024-05-27T12:36:07.390231+0000","last_active":"2024-05-27T12:36:08.411395+0000","last_peered":"2024-05-27T12:36:08.411395+0000","last_clean":"2024-05-27T12:36:08.411395+0000","last_became_active":"2024-05-27T12:36:07.389832+0000","last_became_peered":"2024-05-27T12:36:07.389832+0000","last_unstale":"2024-05-27T12:36:08.411395+0000","last_undegraded":"2024-05-27T12:36:08.411395+0000","last_fullsized":"2024-05-27T12:36:08.411395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:10.994307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406867+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:08.406867+0000","last_peered":"2024-05-27T12:36:08.406867+0000","last_clean":"2024-05-27T12:36:08.406867+0000","last_became_active":"2024-05-27T12:36:06.398355+0000","last_became_peered":"2024-05-27T12:36:06.398355+0000","last_unstale":"2024-05-27T12:36:08.406867+0000","last_undegraded":"2024-05-27T12:36:08.406867+0000","last_fullsized":"2024-05-27T12:36:08.406867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413435+0000","last_change":"2024-05-27T12:36:07.403495+0000","last_active":"2024-05-27T12:36:08.413435+0000","last_peered":"2024-05-27T12:36:08.413435+0000","last_clean":"2024-05-27T12:36:08.413435+0000","last_became_active":"2024-05-27T12:36:07.401246+0000","last_became_peered":"2024-05-27T12:36:07.401246+0000","last_unstale":"2024-05-27T12:36:08.413435+0000","last_undegraded":"2024-05-27T12:36:08.413435+0000","last_fullsized":"2024-05-27T12:36:08.413435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:57:51.876839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500916+0000","last_change":"2024-05-27T12:36:06.393410+0000","last_active":"2024-05-27T12:36:08.500916+0000","last_peered":"2024-05-27T12:36:08.500916+0000","last_clean":"2024-05-27T12:36:08.500916+0000","last_became_active":"2024-05-27T12:36:06.392937+0000","last_became_peered":"2024-05-27T12:36:06.392937+0000","last_unstale":"2024-05-27T12:36:08.500916+0000","last_undegraded":"2024-05-27T12:36:08.500916+0000","last_fullsized":"2024-05-27T12:36:08.500916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:40.771809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421326+0000","last_change":"2024-05-27T12:36:07.411913+0000","last_active":"2024-05-27T12:36:08.421326+0000","last_peered":"2024-05-27T12:36:08.421326+0000","last_clean":"2024-05-27T12:36:08.421326+0000","last_became_active":"2024-05-27T12:36:07.410523+0000","last_became_peered":"2024-05-27T12:36:07.410523+0000","last_unstale":"2024-05-27T12:36:08.421326+0000","last_undegraded":"2024-05-27T12:36:08.421326+0000","last_fullsized":"2024-05-27T12:36:08.421326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:06:56.686640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500613+0000","last_change":"2024-05-27T12:36:06.395813+0000","last_active":"2024-05-27T12:36:08.500613+0000","last_peered":"2024-05-27T12:36:08.500613+0000","last_clean":"2024-05-27T12:36:08.500613+0000","last_became_active":"2024-05-27T12:36:06.392355+0000","last_became_peered":"2024-05-27T12:36:06.392355+0000","last_unstale":"2024-05-27T12:36:08.500613+0000","last_undegraded":"2024-05-27T12:36:08.500613+0000","last_fullsized":"2024-05-27T12:36:08.500613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:11:10.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457667+0000","last_change":"2024-05-27T12:36:07.410252+0000","last_active":"2024-05-27T12:36:08.457667+0000","last_peered":"2024-05-27T12:36:08.457667+0000","last_clean":"2024-05-27T12:36:08.457667+0000","last_became_active":"2024-05-27T12:36:07.409955+0000","last_became_peered":"2024-05-27T12:36:07.409955+0000","last_unstale":"2024-05-27T12:36:08.457667+0000","last_undegraded":"2024-05-27T12:36:08.457667+0000","last_fullsized":"2024-05-27T12:36:08.457667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411888+0000","last_change":"2024-05-27T12:36:06.394640+0000","last_active":"2024-05-27T12:36:08.411888+0000","last_peered":"2024-05-27T12:36:08.411888+0000","last_clean":"2024-05-27T12:36:08.411888+0000","last_became_active":"2024-05-27T12:36:06.394297+0000","last_became_peered":"2024-05-27T12:36:06.394297+0000","last_unstale":"2024-05-27T12:36:08.411888+0000","last_undegraded":"2024-05-27T12:36:08.411888+0000","last_fullsized":"2024-05-27T12:36:08.411888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503216+0000","last_change":"2024-05-27T12:36:07.406862+0000","last_active":"2024-05-27T12:36:08.503216+0000","last_peered":"2024-05-27T12:36:08.503216+0000","last_clean":"2024-05-27T12:36:08.503216+0000","last_became_active":"2024-05-27T12:36:07.404374+0000","last_became_peered":"2024-05-27T12:36:07.404374+0000","last_unstale":"2024-05-27T12:36:08.503216+0000","last_undegraded":"2024-05-27T12:36:08.503216+0000","last_fullsized":"2024-05-27T12:36:08.503216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410782+0000","last_change":"2024-05-27T12:36:06.394969+0000","last_active":"2024-05-27T12:36:08.410782+0000","last_peered":"2024-05-27T12:36:08.410782+0000","last_clean":"2024-05-27T12:36:08.410782+0000","last_became_active":"2024-05-27T12:36:06.394600+0000","last_became_peered":"2024-05-27T12:36:06.394600+0000","last_unstale":"2024-05-27T12:36:08.410782+0000","last_undegraded":"2024-05-27T12:36:08.410782+0000","last_fullsized":"2024-05-27T12:36:08.410782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412644+0000","last_change":"2024-05-27T12:36:07.389401+0000","last_active":"2024-05-27T12:36:08.412644+0000","last_peered":"2024-05-27T12:36:08.412644+0000","last_clean":"2024-05-27T12:36:08.412644+0000","last_became_active":"2024-05-27T12:36:07.388474+0000","last_became_peered":"2024-05-27T12:36:07.388474+0000","last_unstale":"2024-05-27T12:36:08.412644+0000","last_undegraded":"2024-05-27T12:36:08.412644+0000","last_fullsized":"2024-05-27T12:36:08.412644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:23.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500230+0000","last_change":"2024-05-27T12:36:06.383317+0000","last_active":"2024-05-27T12:36:08.500230+0000","last_peered":"2024-05-27T12:36:08.500230+0000","last_clean":"2024-05-27T12:36:08.500230+0000","last_became_active":"2024-05-27T12:36:06.383006+0000","last_became_peered":"2024-05-27T12:36:06.383006+0000","last_unstale":"2024-05-27T12:36:08.500230+0000","last_undegraded":"2024-05-27T12:36:08.500230+0000","last_fullsized":"2024-05-27T12:36:08.500230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:44:22.857477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420729+0000","last_change":"2024-05-27T12:36:07.415779+0000","last_active":"2024-05-27T12:36:08.420729+0000","last_peered":"2024-05-27T12:36:08.420729+0000","last_clean":"2024-05-27T12:36:08.420729+0000","last_became_active":"2024-05-27T12:36:07.415457+0000","last_became_peered":"2024-05-27T12:36:07.415457+0000","last_unstale":"2024-05-27T12:36:08.420729+0000","last_undegraded":"2024-05-27T12:36:08.420729+0000","last_fullsized":"2024-05-27T12:36:08.420729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:57.100241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.409994+0000","last_change":"2024-05-27T12:36:06.398382+0000","last_active":"2024-05-27T12:36:08.409994+0000","last_peered":"2024-05-27T12:36:08.409994+0000","last_clean":"2024-05-27T12:36:08.409994+0000","last_became_active":"2024-05-27T12:36:06.396324+0000","last_became_peered":"2024-05-27T12:36:06.396324+0000","last_unstale":"2024-05-27T12:36:08.409994+0000","last_undegraded":"2024-05-27T12:36:08.409994+0000","last_fullsized":"2024-05-27T12:36:08.409994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504109+0000","last_change":"2024-05-27T12:36:07.402050+0000","last_active":"2024-05-27T12:36:08.504109+0000","last_peered":"2024-05-27T12:36:08.504109+0000","last_clean":"2024-05-27T12:36:08.504109+0000","last_became_active":"2024-05-27T12:36:07.401684+0000","last_became_peered":"2024-05-27T12:36:07.401684+0000","last_unstale":"2024-05-27T12:36:08.504109+0000","last_undegraded":"2024-05-27T12:36:08.504109+0000","last_fullsized":"2024-05-27T12:36:08.504109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:24.704060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500985+0000","last_change":"2024-05-27T12:36:06.400135+0000","last_active":"2024-05-27T12:36:08.500985+0000","last_peered":"2024-05-27T12:36:08.500985+0000","last_clean":"2024-05-27T12:36:08.500985+0000","last_became_active":"2024-05-27T12:36:06.399910+0000","last_became_peered":"2024-05-27T12:36:06.399910+0000","last_unstale":"2024-05-27T12:36:08.500985+0000","last_undegraded":"2024-05-27T12:36:08.500985+0000","last_fullsized":"2024-05-27T12:36:08.500985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:15.544406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501008+0000","last_change":"2024-05-27T12:36:07.413359+0000","last_active":"2024-05-27T12:36:08.501008+0000","last_peered":"2024-05-27T12:36:08.501008+0000","last_clean":"2024-05-27T12:36:08.501008+0000","last_became_active":"2024-05-27T12:36:07.413021+0000","last_became_peered":"2024-05-27T12:36:07.413021+0000","last_unstale":"2024-05-27T12:36:08.501008+0000","last_undegraded":"2024-05-27T12:36:08.501008+0000","last_fullsized":"2024-05-27T12:36:08.501008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406992+0000","last_change":"2024-05-27T12:36:06.383845+0000","last_active":"2024-05-27T12:36:08.406992+0000","last_peered":"2024-05-27T12:36:08.406992+0000","last_clean":"2024-05-27T12:36:08.406992+0000","last_became_active":"2024-05-27T12:36:06.383394+0000","last_became_peered":"2024-05-27T12:36:06.383394+0000","last_unstale":"2024-05-27T12:36:08.406992+0000","last_undegraded":"2024-05-27T12:36:08.406992+0000","last_fullsized":"2024-05-27T12:36:08.406992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501478+0000","last_change":"2024-05-27T12:36:07.397487+0000","last_active":"2024-05-27T12:36:08.501478+0000","last_peered":"2024-05-27T12:36:08.501478+0000","last_clean":"2024-05-27T12:36:08.501478+0000","last_became_active":"2024-05-27T12:36:07.397135+0000","last_became_peered":"2024-05-27T12:36:07.397135+0000","last_unstale":"2024-05-27T12:36:08.501478+0000","last_undegraded":"2024-05-27T12:36:08.501478+0000","last_fullsized":"2024-05-27T12:36:08.501478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504250+0000","last_change":"2024-05-27T12:36:06.398488+0000","last_active":"2024-05-27T12:36:08.504250+0000","last_peered":"2024-05-27T12:36:08.504250+0000","last_clean":"2024-05-27T12:36:08.504250+0000","last_became_active":"2024-05-27T12:36:06.397511+0000","last_became_peered":"2024-05-27T12:36:06.397511+0000","last_unstale":"2024-05-27T12:36:08.504250+0000","last_undegraded":"2024-05-27T12:36:08.504250+0000","last_fullsized":"2024-05-27T12:36:08.504250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:56.087700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503992+0000","last_change":"2024-05-27T12:36:07.407283+0000","last_active":"2024-05-27T12:36:08.503992+0000","last_peered":"2024-05-27T12:36:08.503992+0000","last_clean":"2024-05-27T12:36:08.503992+0000","last_became_active":"2024-05-27T12:36:07.407020+0000","last_became_peered":"2024-05-27T12:36:07.407020+0000","last_unstale":"2024-05-27T12:36:08.503992+0000","last_undegraded":"2024-05-27T12:36:08.503992+0000","last_fullsized":"2024-05-27T12:36:08.503992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501461+0000","last_change":"2024-05-27T12:36:06.393888+0000","last_active":"2024-05-27T12:36:08.501461+0000","last_peered":"2024-05-27T12:36:08.501461+0000","last_clean":"2024-05-27T12:36:08.501461+0000","last_became_active":"2024-05-27T12:36:06.393567+0000","last_became_peered":"2024-05-27T12:36:06.393567+0000","last_unstale":"2024-05-27T12:36:08.501461+0000","last_undegraded":"2024-05-27T12:36:08.501461+0000","last_fullsized":"2024-05-27T12:36:08.501461+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:03.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420762+0000","last_change":"2024-05-27T12:36:07.418706+0000","last_active":"2024-05-27T12:36:08.420762+0000","last_peered":"2024-05-27T12:36:08.420762+0000","last_clean":"2024-05-27T12:36:08.420762+0000","last_became_active":"2024-05-27T12:36:07.416238+0000","last_became_peered":"2024-05-27T12:36:07.416238+0000","last_unstale":"2024-05-27T12:36:08.420762+0000","last_undegraded":"2024-05-27T12:36:08.420762+0000","last_fullsized":"2024-05-27T12:36:08.420762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:23.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412195+0000","last_change":"2024-05-27T12:36:06.396297+0000","last_active":"2024-05-27T12:36:08.412195+0000","last_peered":"2024-05-27T12:36:08.412195+0000","last_clean":"2024-05-27T12:36:08.412195+0000","last_became_active":"2024-05-27T12:36:06.395605+0000","last_became_peered":"2024-05-27T12:36:06.395605+0000","last_unstale":"2024-05-27T12:36:08.412195+0000","last_undegraded":"2024-05-27T12:36:08.412195+0000","last_fullsized":"2024-05-27T12:36:08.412195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500848+0000","last_change":"2024-05-27T12:36:07.400200+0000","last_active":"2024-05-27T12:36:08.500848+0000","last_peered":"2024-05-27T12:36:08.500848+0000","last_clean":"2024-05-27T12:36:08.500848+0000","last_became_active":"2024-05-27T12:36:07.399899+0000","last_became_peered":"2024-05-27T12:36:07.399899+0000","last_unstale":"2024-05-27T12:36:08.500848+0000","last_undegraded":"2024-05-27T12:36:08.500848+0000","last_fullsized":"2024-05-27T12:36:08.500848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420802+0000","last_change":"2024-05-27T12:36:06.391699+0000","last_active":"2024-05-27T12:36:08.420802+0000","last_peered":"2024-05-27T12:36:08.420802+0000","last_clean":"2024-05-27T12:36:08.420802+0000","last_became_active":"2024-05-27T12:36:06.391285+0000","last_became_peered":"2024-05-27T12:36:06.391285+0000","last_unstale":"2024-05-27T12:36:08.420802+0000","last_undegraded":"2024-05-27T12:36:08.420802+0000","last_fullsized":"2024-05-27T12:36:08.420802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458217+0000","last_change":"2024-05-27T12:36:07.406688+0000","last_active":"2024-05-27T12:36:08.458217+0000","last_peered":"2024-05-27T12:36:08.458217+0000","last_clean":"2024-05-27T12:36:08.458217+0000","last_became_active":"2024-05-27T12:36:07.405915+0000","last_became_peered":"2024-05-27T12:36:07.405915+0000","last_unstale":"2024-05-27T12:36:08.458217+0000","last_undegraded":"2024-05-27T12:36:08.458217+0000","last_fullsized":"2024-05-27T12:36:08.458217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:39.937707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500135+0000","last_change":"2024-05-27T12:36:06.399046+0000","last_active":"2024-05-27T12:36:08.500135+0000","last_peered":"2024-05-27T12:36:08.500135+0000","last_clean":"2024-05-27T12:36:08.500135+0000","last_became_active":"2024-05-27T12:36:06.396837+0000","last_became_peered":"2024-05-27T12:36:06.396837+0000","last_unstale":"2024-05-27T12:36:08.500135+0000","last_undegraded":"2024-05-27T12:36:08.500135+0000","last_fullsized":"2024-05-27T12:36:08.500135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:09.981766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411528+0000","last_change":"2024-05-27T12:36:07.400590+0000","last_active":"2024-05-27T12:36:08.411528+0000","last_peered":"2024-05-27T12:36:08.411528+0000","last_clean":"2024-05-27T12:36:08.411528+0000","last_became_active":"2024-05-27T12:36:07.399770+0000","last_became_peered":"2024-05-27T12:36:07.399770+0000","last_unstale":"2024-05-27T12:36:08.411528+0000","last_undegraded":"2024-05-27T12:36:08.411528+0000","last_fullsized":"2024-05-27T12:36:08.411528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:44:23.870018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500114+0000","last_change":"2024-05-27T12:36:06.397002+0000","last_active":"2024-05-27T12:36:08.500114+0000","last_peered":"2024-05-27T12:36:08.500114+0000","last_clean":"2024-05-27T12:36:08.500114+0000","last_became_active":"2024-05-27T12:36:06.396513+0000","last_became_peered":"2024-05-27T12:36:06.396513+0000","last_unstale":"2024-05-27T12:36:08.500114+0000","last_undegraded":"2024-05-27T12:36:08.500114+0000","last_fullsized":"2024-05-27T12:36:08.500114+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:23.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.414036+0000","last_change":"2024-05-27T12:36:07.409445+0000","last_active":"2024-05-27T12:36:08.414036+0000","last_peered":"2024-05-27T12:36:08.414036+0000","last_clean":"2024-05-27T12:36:08.414036+0000","last_became_active":"2024-05-27T12:36:07.409162+0000","last_became_peered":"2024-05-27T12:36:07.409162+0000","last_unstale":"2024-05-27T12:36:08.414036+0000","last_undegraded":"2024-05-27T12:36:08.414036+0000","last_fullsized":"2024-05-27T12:36:08.414036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:13.677115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421487+0000","last_change":"2024-05-27T12:36:06.390945+0000","last_active":"2024-05-27T12:36:08.421487+0000","last_peered":"2024-05-27T12:36:08.421487+0000","last_clean":"2024-05-27T12:36:08.421487+0000","last_became_active":"2024-05-27T12:36:06.390575+0000","last_became_peered":"2024-05-27T12:36:06.390575+0000","last_unstale":"2024-05-27T12:36:08.421487+0000","last_undegraded":"2024-05-27T12:36:08.421487+0000","last_fullsized":"2024-05-27T12:36:08.421487+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504032+0000","last_change":"2024-05-27T12:36:07.413660+0000","last_active":"2024-05-27T12:36:08.504032+0000","last_peered":"2024-05-27T12:36:08.504032+0000","last_clean":"2024-05-27T12:36:08.504032+0000","last_became_active":"2024-05-27T12:36:07.413196+0000","last_became_peered":"2024-05-27T12:36:07.413196+0000","last_unstale":"2024-05-27T12:36:08.504032+0000","last_undegraded":"2024-05-27T12:36:08.504032+0000","last_fullsized":"2024-05-27T12:36:08.504032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413769+0000","last_change":"2024-05-27T12:36:06.396687+0000","last_active":"2024-05-27T12:36:08.413769+0000","last_peered":"2024-05-27T12:36:08.413769+0000","last_clean":"2024-05-27T12:36:08.413769+0000","last_became_active":"2024-05-27T12:36:06.395340+0000","last_became_peered":"2024-05-27T12:36:06.395340+0000","last_unstale":"2024-05-27T12:36:08.413769+0000","last_undegraded":"2024-05-27T12:36:08.413769+0000","last_fullsized":"2024-05-27T12:36:08.413769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421041+0000","last_change":"2024-05-27T12:36:07.417147+0000","last_active":"2024-05-27T12:36:08.421041+0000","last_peered":"2024-05-27T12:36:08.421041+0000","last_clean":"2024-05-27T12:36:08.421041+0000","last_became_active":"2024-05-27T12:36:07.416535+0000","last_became_peered":"2024-05-27T12:36:07.416535+0000","last_unstale":"2024-05-27T12:36:08.421041+0000","last_undegraded":"2024-05-27T12:36:08.421041+0000","last_fullsized":"2024-05-27T12:36:08.421041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:57:51.876839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413707+0000","last_change":"2024-05-27T12:36:06.393791+0000","last_active":"2024-05-27T12:36:08.413707+0000","last_peered":"2024-05-27T12:36:08.413707+0000","last_clean":"2024-05-27T12:36:08.413707+0000","last_became_active":"2024-05-27T12:36:06.391768+0000","last_became_peered":"2024-05-27T12:36:06.391768+0000","last_unstale":"2024-05-27T12:36:08.413707+0000","last_undegraded":"2024-05-27T12:36:08.413707+0000","last_fullsized":"2024-05-27T12:36:08.413707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500593+0000","last_change":"2024-05-27T12:36:07.399674+0000","last_active":"2024-05-27T12:36:08.500593+0000","last_peered":"2024-05-27T12:36:08.500593+0000","last_clean":"2024-05-27T12:36:08.500593+0000","last_became_active":"2024-05-27T12:36:07.399374+0000","last_became_peered":"2024-05-27T12:36:07.399374+0000","last_unstale":"2024-05-27T12:36:08.500593+0000","last_undegraded":"2024-05-27T12:36:08.500593+0000","last_fullsized":"2024-05-27T12:36:08.500593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:22.574391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413277+0000","last_change":"2024-05-27T12:36:06.388667+0000","last_active":"2024-05-27T12:36:08.413277+0000","last_peered":"2024-05-27T12:36:08.413277+0000","last_clean":"2024-05-27T12:36:08.413277+0000","last_became_active":"2024-05-27T12:36:06.388369+0000","last_became_peered":"2024-05-27T12:36:06.388369+0000","last_unstale":"2024-05-27T12:36:08.413277+0000","last_undegraded":"2024-05-27T12:36:08.413277+0000","last_fullsized":"2024-05-27T12:36:08.413277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500390+0000","last_change":"2024-05-27T12:36:07.392615+0000","last_active":"2024-05-27T12:36:08.500390+0000","last_peered":"2024-05-27T12:36:08.500390+0000","last_clean":"2024-05-27T12:36:08.500390+0000","last_became_active":"2024-05-27T12:36:07.392259+0000","last_became_peered":"2024-05-27T12:36:07.392259+0000","last_unstale":"2024-05-27T12:36:08.500390+0000","last_undegraded":"2024-05-27T12:36:08.500390+0000","last_fullsized":"2024-05-27T12:36:08.500390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:19.866522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406813+0000","last_change":"2024-05-27T12:36:06.391794+0000","last_active":"2024-05-27T12:36:08.406813+0000","last_peered":"2024-05-27T12:36:08.406813+0000","last_clean":"2024-05-27T12:36:08.406813+0000","last_became_active":"2024-05-27T12:36:06.389977+0000","last_became_peered":"2024-05-27T12:36:06.389977+0000","last_unstale":"2024-05-27T12:36:08.406813+0000","last_undegraded":"2024-05-27T12:36:08.406813+0000","last_fullsized":"2024-05-27T12:36:08.406813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500662+0000","last_change":"2024-05-27T12:36:07.409529+0000","last_active":"2024-05-27T12:36:08.500662+0000","last_peered":"2024-05-27T12:36:08.500662+0000","last_clean":"2024-05-27T12:36:08.500662+0000","last_became_active":"2024-05-27T12:36:07.404579+0000","last_became_peered":"2024-05-27T12:36:07.404579+0000","last_unstale":"2024-05-27T12:36:08.500662+0000","last_undegraded":"2024-05-27T12:36:08.500662+0000","last_fullsized":"2024-05-27T12:36:08.500662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:12.971444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412145+0000","last_change":"2024-05-27T12:36:06.393310+0000","last_active":"2024-05-27T12:36:08.412145+0000","last_peered":"2024-05-27T12:36:08.412145+0000","last_clean":"2024-05-27T12:36:08.412145+0000","last_became_active":"2024-05-27T12:36:06.392014+0000","last_became_peered":"2024-05-27T12:36:06.392014+0000","last_unstale":"2024-05-27T12:36:08.412145+0000","last_undegraded":"2024-05-27T12:36:08.412145+0000","last_fullsized":"2024-05-27T12:36:08.412145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500302+0000","last_change":"2024-05-27T12:36:07.407681+0000","last_active":"2024-05-27T12:36:08.500302+0000","last_peered":"2024-05-27T12:36:08.500302+0000","last_clean":"2024-05-27T12:36:08.500302+0000","last_became_active":"2024-05-27T12:36:07.407404+0000","last_became_peered":"2024-05-27T12:36:07.407404+0000","last_unstale":"2024-05-27T12:36:08.500302+0000","last_undegraded":"2024-05-27T12:36:08.500302+0000","last_fullsized":"2024-05-27T12:36:08.500302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:18.345354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412472+0000","last_change":"2024-05-27T12:36:06.393400+0000","last_active":"2024-05-27T12:36:08.412472+0000","last_peered":"2024-05-27T12:36:08.412472+0000","last_clean":"2024-05-27T12:36:08.412472+0000","last_became_active":"2024-05-27T12:36:06.392996+0000","last_became_peered":"2024-05-27T12:36:06.392996+0000","last_unstale":"2024-05-27T12:36:08.412472+0000","last_undegraded":"2024-05-27T12:36:08.412472+0000","last_fullsized":"2024-05-27T12:36:08.412472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458016+0000","last_change":"2024-05-27T12:36:07.408808+0000","last_active":"2024-05-27T12:36:08.458016+0000","last_peered":"2024-05-27T12:36:08.458016+0000","last_clean":"2024-05-27T12:36:08.458016+0000","last_became_active":"2024-05-27T12:36:07.406710+0000","last_became_peered":"2024-05-27T12:36:07.406710+0000","last_unstale":"2024-05-27T12:36:08.458016+0000","last_undegraded":"2024-05-27T12:36:08.458016+0000","last_fullsized":"2024-05-27T12:36:08.458016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.502889+0000","last_change":"2024-05-27T12:36:06.396022+0000","last_active":"2024-05-27T12:36:08.502889+0000","last_peered":"2024-05-27T12:36:08.502889+0000","last_clean":"2024-05-27T12:36:08.502889+0000","last_became_active":"2024-05-27T12:36:06.395727+0000","last_became_peered":"2024-05-27T12:36:06.395727+0000","last_unstale":"2024-05-27T12:36:08.502889+0000","last_undegraded":"2024-05-27T12:36:08.502889+0000","last_fullsized":"2024-05-27T12:36:08.502889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:48:06.291144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500734+0000","last_change":"2024-05-27T12:36:07.410480+0000","last_active":"2024-05-27T12:36:08.500734+0000","last_peered":"2024-05-27T12:36:08.500734+0000","last_clean":"2024-05-27T12:36:08.500734+0000","last_became_active":"2024-05-27T12:36:07.406987+0000","last_became_peered":"2024-05-27T12:36:07.406987+0000","last_unstale":"2024-05-27T12:36:08.500734+0000","last_undegraded":"2024-05-27T12:36:08.500734+0000","last_fullsized":"2024-05-27T12:36:08.500734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:39.961835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421551+0000","last_change":"2024-05-27T12:36:06.393032+0000","last_active":"2024-05-27T12:36:08.421551+0000","last_peered":"2024-05-27T12:36:08.421551+0000","last_clean":"2024-05-27T12:36:08.421551+0000","last_became_active":"2024-05-27T12:36:06.392754+0000","last_became_peered":"2024-05-27T12:36:06.392754+0000","last_unstale":"2024-05-27T12:36:08.421551+0000","last_undegraded":"2024-05-27T12:36:08.421551+0000","last_fullsized":"2024-05-27T12:36:08.421551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504031+0000","last_change":"2024-05-27T12:36:07.408023+0000","last_active":"2024-05-27T12:36:08.504031+0000","last_peered":"2024-05-27T12:36:08.504031+0000","last_clean":"2024-05-27T12:36:08.504031+0000","last_became_active":"2024-05-27T12:36:07.407695+0000","last_became_peered":"2024-05-27T12:36:07.407695+0000","last_unstale":"2024-05-27T12:36:08.504031+0000","last_undegraded":"2024-05-27T12:36:08.504031+0000","last_fullsized":"2024-05-27T12:36:08.504031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:39.937707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406962+0000","last_change":"2024-05-27T12:36:06.399546+0000","last_active":"2024-05-27T12:36:08.406962+0000","last_peered":"2024-05-27T12:36:08.406962+0000","last_clean":"2024-05-27T12:36:08.406962+0000","last_became_active":"2024-05-27T12:36:06.399228+0000","last_became_peered":"2024-05-27T12:36:06.399228+0000","last_unstale":"2024-05-27T12:36:08.406962+0000","last_undegraded":"2024-05-27T12:36:08.406962+0000","last_fullsized":"2024-05-27T12:36:08.406962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406980+0000","last_change":"2024-05-27T12:36:07.405026+0000","last_active":"2024-05-27T12:36:08.406980+0000","last_peered":"2024-05-27T12:36:08.406980+0000","last_clean":"2024-05-27T12:36:08.406980+0000","last_became_active":"2024-05-27T12:36:07.404731+0000","last_became_peered":"2024-05-27T12:36:07.404731+0000","last_unstale":"2024-05-27T12:36:08.406980+0000","last_undegraded":"2024-05-27T12:36:08.406980+0000","last_fullsized":"2024-05-27T12:36:08.406980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:16.556947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421180+0000","last_change":"2024-05-27T12:36:06.391239+0000","last_active":"2024-05-27T12:36:08.421180+0000","last_peered":"2024-05-27T12:36:08.421180+0000","last_clean":"2024-05-27T12:36:08.421180+0000","last_became_active":"2024-05-27T12:36:06.390800+0000","last_became_peered":"2024-05-27T12:36:06.390800+0000","last_unstale":"2024-05-27T12:36:08.421180+0000","last_undegraded":"2024-05-27T12:36:08.421180+0000","last_fullsized":"2024-05-27T12:36:08.421180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:38.949294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412409+0000","last_change":"2024-05-27T12:36:07.390309+0000","last_active":"2024-05-27T12:36:08.412409+0000","last_peered":"2024-05-27T12:36:08.412409+0000","last_clean":"2024-05-27T12:36:08.412409+0000","last_became_active":"2024-05-27T12:36:07.389963+0000","last_became_peered":"2024-05-27T12:36:07.389963+0000","last_unstale":"2024-05-27T12:36:08.412409+0000","last_undegraded":"2024-05-27T12:36:08.412409+0000","last_fullsized":"2024-05-27T12:36:08.412409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:24.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.407071+0000","last_change":"2024-05-27T12:36:06.389001+0000","last_active":"2024-05-27T12:36:08.407071+0000","last_peered":"2024-05-27T12:36:08.407071+0000","last_clean":"2024-05-27T12:36:08.407071+0000","last_became_active":"2024-05-27T12:36:06.388700+0000","last_became_peered":"2024-05-27T12:36:06.388700+0000","last_unstale":"2024-05-27T12:36:08.407071+0000","last_undegraded":"2024-05-27T12:36:08.407071+0000","last_fullsized":"2024-05-27T12:36:08.407071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406900+0000","last_change":"2024-05-27T12:36:07.392244+0000","last_active":"2024-05-27T12:36:08.406900+0000","last_peered":"2024-05-27T12:36:08.406900+0000","last_clean":"2024-05-27T12:36:08.406900+0000","last_became_active":"2024-05-27T12:36:07.391822+0000","last_became_peered":"2024-05-27T12:36:07.391822+0000","last_unstale":"2024-05-27T12:36:08.406900+0000","last_undegraded":"2024-05-27T12:36:08.406900+0000","last_fullsized":"2024-05-27T12:36:08.406900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:04.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420901+0000","last_change":"2024-05-27T12:36:06.390271+0000","last_active":"2024-05-27T12:36:08.420901+0000","last_peered":"2024-05-27T12:36:08.420901+0000","last_clean":"2024-05-27T12:36:08.420901+0000","last_became_active":"2024-05-27T12:36:06.389933+0000","last_became_peered":"2024-05-27T12:36:06.389933+0000","last_unstale":"2024-05-27T12:36:08.420901+0000","last_undegraded":"2024-05-27T12:36:08.420901+0000","last_fullsized":"2024-05-27T12:36:08.420901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.499988+0000","last_change":"2024-05-27T12:36:07.403635+0000","last_active":"2024-05-27T12:36:08.499988+0000","last_peered":"2024-05-27T12:36:08.499988+0000","last_clean":"2024-05-27T12:36:08.499988+0000","last_became_active":"2024-05-27T12:36:07.403356+0000","last_became_peered":"2024-05-27T12:36:07.403356+0000","last_unstale":"2024-05-27T12:36:08.499988+0000","last_undegraded":"2024-05-27T12:36:08.499988+0000","last_fullsized":"2024-05-27T12:36:08.499988+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:05.585799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412719+0000","last_change":"2024-05-27T12:36:06.389847+0000","last_active":"2024-05-27T12:36:08.412719+0000","last_peered":"2024-05-27T12:36:08.412719+0000","last_clean":"2024-05-27T12:36:08.412719+0000","last_became_active":"2024-05-27T12:36:06.389431+0000","last_became_peered":"2024-05-27T12:36:06.389431+0000","last_unstale":"2024-05-27T12:36:08.412719+0000","last_undegraded":"2024-05-27T12:36:08.412719+0000","last_fullsized":"2024-05-27T12:36:08.412719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412201+0000","last_change":"2024-05-27T12:36:07.395868+0000","last_active":"2024-05-27T12:36:08.412201+0000","last_peered":"2024-05-27T12:36:08.412201+0000","last_clean":"2024-05-27T12:36:08.412201+0000","last_became_active":"2024-05-27T12:36:07.395186+0000","last_became_peered":"2024-05-27T12:36:07.395186+0000","last_unstale":"2024-05-27T12:36:08.412201+0000","last_undegraded":"2024-05-27T12:36:08.412201+0000","last_fullsized":"2024-05-27T12:36:08.412201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:57.100241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.499993+0000","last_change":"2024-05-27T12:36:06.399243+0000","last_active":"2024-05-27T12:36:08.499993+0000","last_peered":"2024-05-27T12:36:08.499993+0000","last_clean":"2024-05-27T12:36:08.499993+0000","last_became_active":"2024-05-27T12:36:06.397252+0000","last_became_peered":"2024-05-27T12:36:06.397252+0000","last_unstale":"2024-05-27T12:36:08.499993+0000","last_undegraded":"2024-05-27T12:36:08.499993+0000","last_fullsized":"2024-05-27T12:36:08.499993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:33:11.307403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457795+0000","last_change":"2024-05-27T12:36:07.403646+0000","last_active":"2024-05-27T12:36:08.457795+0000","last_peered":"2024-05-27T12:36:08.457795+0000","last_clean":"2024-05-27T12:36:08.457795+0000","last_became_active":"2024-05-27T12:36:07.403383+0000","last_became_peered":"2024-05-27T12:36:07.403383+0000","last_unstale":"2024-05-27T12:36:08.457795+0000","last_undegraded":"2024-05-27T12:36:08.457795+0000","last_fullsized":"2024-05-27T12:36:08.457795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:49:21.153367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412305+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:08.412305+0000","last_peered":"2024-05-27T12:36:08.412305+0000","last_clean":"2024-05-27T12:36:08.412305+0000","last_became_active":"2024-05-27T12:36:06.398205+0000","last_became_peered":"2024-05-27T12:36:06.398205+0000","last_unstale":"2024-05-27T12:36:08.412305+0000","last_undegraded":"2024-05-27T12:36:08.412305+0000","last_fullsized":"2024-05-27T12:36:08.412305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458456+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:36:08.458456+0000","last_peered":"2024-05-27T12:36:08.458456+0000","last_clean":"2024-05-27T12:36:08.458456+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:36:08.458456+0000","last_undegraded":"2024-05-27T12:36:08.458456+0000","last_fullsized":"2024-05-27T12:36:08.458456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:36:16.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T12:36:17.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.019+0000 7f2b8fc00640 1 Processor -- start 2024-05-27T12:36:17.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.019+0000 7f2b8fc00640 1 -- start start 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.020+0000 7f2b8fc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b8815a250 0x7f2b8806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.020+0000 7f2b8fc00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b88160360 con 0x7f2b8806f280 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b8fc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b88160640 con 0x7f2b880cfb10 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b8fc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2b88160f70 con 0x7f2b8815a250 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b7fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2b8806f280 0x7f2b880cf400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36598/0 (socket says 172.21.15.94:36598) 2024-05-27T12:36:17.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b7fe00640 1 -- 172.21.15.94:0/1339970757 learned_addr learned my addr 172.21.15.94:0/1339970757 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:17.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b8d200640 1 --2- 172.21.15.94:0/1339970757 >> v2:172.21.15.97:3300/0 conn(0x7f2b8815a250 0x7f2b8806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853229678 0 0) 0x7f2b88160360 con 0x7f2b8806f280 2024-05-27T12:36:17.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.021+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b6c003520 con 0x7f2b8806f280 2024-05-27T12:36:17.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 180001343 0 0) 0x7f2b6c003520 con 0x7f2b8806f280 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 >> v2:172.21.15.97:3300/0 conn(0x7f2b8815a250 msgr2=0x7f2b8806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 --2- 172.21.15.94:0/1339970757 >> v2:172.21.15.97:3300/0 conn(0x7f2b8815a250 0x7f2b8806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 >> v1:172.21.15.97:6789/0 conn(0x7f2b880cfb10 legacy=0x7f2b880c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b88160920 con 0x7f2b8806f280 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f2b70001670 con 0x7f2b8806f280 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 >> v1:172.21.15.94:6789/0 conn(0x7f2b8806f280 legacy=0x7f2b880cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c003ae0 0x7f2b6c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c004800 0x7f2b6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 0x7f2b6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00c160 con 0x7f2b6c0052e0 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00c310 con 0x7f2b6c004800 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c00c510 con 0x7f2b6c003ae0 2024-05-27T12:36:17.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.022+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 0x7f2b6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b7fe00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c004800 0x7f2b6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8d200640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c003ae0 0x7f2b6c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8dc00640 1 -- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c003ae0 msgr2=0x7f2b6c003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c003ae0 0x7f2b6c003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8dc00640 1 -- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c004800 msgr2=0x7f2b6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c004800 0x7f2b6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b8dc00640 1 -- 172.21.15.94:0/1339970757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c003520 con 0x7f2b6c0052e0 2024-05-27T12:36:17.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.023+0000 7f2b7fe00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c004800 0x7f2b6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:36:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.024+0000 7f2b8dc00640 1 -- 172.21.15.94:0/1339970757 reap_dead start 2024-05-27T12:36:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.024+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 0x7f2b6c00b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2b88160f20 tx=0x7f2b6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e57c3d14e94887d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.024+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80001a70 con 0x7f2b6c0052e0 2024-05-27T12:36:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.024+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b8002ae10 con 0x7f2b6c0052e0 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.024+0000 7f2b87e00640 1 -- 172.21.15.94:0/1339970757 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80003bb0 con 0x7f2b6c0052e0 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 -- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 msgr2=0x7f2b6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 0x7f2b6c00b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2b88160f20 tx=0x7f2b6c005830 comp rx=0 tx=0).stop 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 -- 172.21.15.94:0/1339970757 shutdown_connections 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/1339970757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b6c0052e0 0x7f2b6c00b9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 -- 172.21.15.94:0/1339970757 >> 172.21.15.94:0/1339970757 conn(0x7f2b880b3830 msgr2=0x7f2b880b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 -- 172.21.15.94:0/1339970757 shutdown_connections 2024-05-27T12:36:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.025+0000 7f2b8fc00640 1 -- 172.21.15.94:0/1339970757 wait complete. 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.027+0000 7f2b8fc00640 1 Processor -- start 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.027+0000 7f2b8fc00640 1 -- start start 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c0052e0 0x7f2b881433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 0x7f2b8815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8806e7f0 con 0x7f2b88158030 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8815a630 con 0x7f2b8815b930 2024-05-27T12:36:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8fc00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8806e970 con 0x7f2b6c0052e0 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b7fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b7fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46298/0 (socket says 172.21.15.94:46298) 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b7fe00640 1 -- 172.21.15.94:0/2187973072 learned_addr learned my addr 172.21.15.94:0/2187973072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.028+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 0x7f2b8815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 -- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c0052e0 msgr2=0x7f2b881433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c0052e0 0x7f2b881433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 -- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 msgr2=0x7f2b8815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 0x7f2b8815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 -- 172.21.15.94:0/2187973072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b880cd5f0 con 0x7f2b88158030 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b8dc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 0x7f2b8815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b7fe00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e790 tx=0x7f2b7000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70019070 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7000ee40 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000ca70 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b88158b60 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.029+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b88140900 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.031+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b88075460 con 0x7f2b88158030 2024-05-27T12:36:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.032+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b7001c070 con 0x7f2b88158030 2024-05-27T12:36:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.032+0000 7f2b85600640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.032+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2b7008b1c0 con 0x7f2b88158030 2024-05-27T12:36:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.033+0000 7f2b8d200640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.034+0000 7f2b8d200640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 0x7f2b6406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009890 tx=0x7f2b78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.037+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b70058f90 con 0x7f2b88158030 2024-05-27T12:36:17.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.234+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b88159080 con 0x7f2b6406c8f0 2024-05-27T12:36:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.253+0000 7f2b85600640 1 -- 172.21.15.94:0/2187973072 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f2b88159080 con 0x7f2b6406c8f0 2024-05-27T12:36:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 msgr2=0x7f2b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 0x7f2b6406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009890 tx=0x7f2b78008040 comp rx=0 tx=0).stop 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 msgr2=0x7f2b881584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e790 tx=0x7f2b7000ec50 comp rx=0 tx=0).stop 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 shutdown_connections 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2b6406c8f0 0x7f2b6406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8815b930 0x7f2b8815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88158030 0x7f2b881584b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 --2- 172.21.15.94:0/2187973072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c0052e0 0x7f2b881433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.259+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 >> 172.21.15.94:0/2187973072 conn(0x7f2b880b3830 msgr2=0x7f2b88071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.260+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 shutdown_connections 2024-05-27T12:36:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.260+0000 7f2b8fc00640 1 -- 172.21.15.94:0/2187973072 wait complete. 2024-05-27T12:36:17.279 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-05-27T12:36:16.886048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236952,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737768,"statfs":{"total":773094113280,"available":772851474432,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.517924"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410969+0000","last_change":"2024-05-27T12:36:07.410152+0000","last_active":"2024-05-27T12:36:08.410969+0000","last_peered":"2024-05-27T12:36:08.410969+0000","last_clean":"2024-05-27T12:36:08.410969+0000","last_became_active":"2024-05-27T12:36:07.407363+0000","last_became_peered":"2024-05-27T12:36:07.407363+0000","last_unstale":"2024-05-27T12:36:08.410969+0000","last_undegraded":"2024-05-27T12:36:08.410969+0000","last_fullsized":"2024-05-27T12:36:08.410969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:45:47.782240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501165+0000","last_change":"2024-05-27T12:36:07.400013+0000","last_active":"2024-05-27T12:36:08.501165+0000","last_peered":"2024-05-27T12:36:08.501165+0000","last_clean":"2024-05-27T12:36:08.501165+0000","last_became_active":"2024-05-27T12:36:07.398865+0000","last_became_peered":"2024-05-27T12:36:07.398865+0000","last_unstale":"2024-05-27T12:36:08.501165+0000","last_undegraded":"2024-05-27T12:36:08.501165+0000","last_fullsized":"2024-05-27T12:36:08.501165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504067+0000","last_change":"2024-05-27T12:36:07.405997+0000","last_active":"2024-05-27T12:36:08.504067+0000","last_peered":"2024-05-27T12:36:08.504067+0000","last_clean":"2024-05-27T12:36:08.504067+0000","last_became_active":"2024-05-27T12:36:07.405695+0000","last_became_peered":"2024-05-27T12:36:07.405695+0000","last_unstale":"2024-05-27T12:36:08.504067+0000","last_undegraded":"2024-05-27T12:36:08.504067+0000","last_fullsized":"2024-05-27T12:36:08.504067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:49:21.153367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421039+0000","last_change":"2024-05-27T12:36:07.419031+0000","last_active":"2024-05-27T12:36:08.421039+0000","last_peered":"2024-05-27T12:36:08.421039+0000","last_clean":"2024-05-27T12:36:08.421039+0000","last_became_active":"2024-05-27T12:36:07.415654+0000","last_became_peered":"2024-05-27T12:36:07.415654+0000","last_unstale":"2024-05-27T12:36:08.421039+0000","last_undegraded":"2024-05-27T12:36:08.421039+0000","last_fullsized":"2024-05-27T12:36:08.421039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:13.677115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500345+0000","last_change":"2024-05-27T12:36:07.398198+0000","last_active":"2024-05-27T12:36:08.500345+0000","last_peered":"2024-05-27T12:36:08.500345+0000","last_clean":"2024-05-27T12:36:08.500345+0000","last_became_active":"2024-05-27T12:36:07.397900+0000","last_became_peered":"2024-05-27T12:36:07.397900+0000","last_unstale":"2024-05-27T12:36:08.500345+0000","last_undegraded":"2024-05-27T12:36:08.500345+0000","last_fullsized":"2024-05-27T12:36:08.500345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:24.314862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500556+0000","last_change":"2024-05-27T12:36:07.406235+0000","last_active":"2024-05-27T12:36:08.500556+0000","last_peered":"2024-05-27T12:36:08.500556+0000","last_clean":"2024-05-27T12:36:08.500556+0000","last_became_active":"2024-05-27T12:36:07.405940+0000","last_became_peered":"2024-05-27T12:36:07.405940+0000","last_unstale":"2024-05-27T12:36:08.500556+0000","last_undegraded":"2024-05-27T12:36:08.500556+0000","last_fullsized":"2024-05-27T12:36:08.500556+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:56:42.538427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457814+0000","last_change":"2024-05-27T12:36:07.409550+0000","last_active":"2024-05-27T12:36:08.457814+0000","last_peered":"2024-05-27T12:36:08.457814+0000","last_clean":"2024-05-27T12:36:08.457814+0000","last_became_active":"2024-05-27T12:36:07.408848+0000","last_became_peered":"2024-05-27T12:36:07.408848+0000","last_unstale":"2024-05-27T12:36:08.457814+0000","last_undegraded":"2024-05-27T12:36:08.457814+0000","last_fullsized":"2024-05-27T12:36:08.457814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457537+0000","last_change":"2024-05-27T12:36:07.404370+0000","last_active":"2024-05-27T12:36:08.457537+0000","last_peered":"2024-05-27T12:36:08.457537+0000","last_clean":"2024-05-27T12:36:08.457537+0000","last_became_active":"2024-05-27T12:36:07.403997+0000","last_became_peered":"2024-05-27T12:36:07.403997+0000","last_unstale":"2024-05-27T12:36:08.457537+0000","last_undegraded":"2024-05-27T12:36:08.457537+0000","last_fullsized":"2024-05-27T12:36:08.457537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.414120+0000","last_change":"2024-05-27T12:36:07.401707+0000","last_active":"2024-05-27T12:36:08.414120+0000","last_peered":"2024-05-27T12:36:08.414120+0000","last_clean":"2024-05-27T12:36:08.414120+0000","last_became_active":"2024-05-27T12:36:07.401164+0000","last_became_peered":"2024-05-27T12:36:07.401164+0000","last_unstale":"2024-05-27T12:36:08.414120+0000","last_undegraded":"2024-05-27T12:36:08.414120+0000","last_fullsized":"2024-05-27T12:36:08.414120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:49.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421210+0000","last_change":"2024-05-27T12:36:07.412627+0000","last_active":"2024-05-27T12:36:08.421210+0000","last_peered":"2024-05-27T12:36:08.421210+0000","last_clean":"2024-05-27T12:36:08.421210+0000","last_became_active":"2024-05-27T12:36:07.412376+0000","last_became_peered":"2024-05-27T12:36:07.412376+0000","last_unstale":"2024-05-27T12:36:08.421210+0000","last_undegraded":"2024-05-27T12:36:08.421210+0000","last_fullsized":"2024-05-27T12:36:08.421210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:49.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458107+0000","last_change":"2024-05-27T12:36:07.405204+0000","last_active":"2024-05-27T12:36:08.458107+0000","last_peered":"2024-05-27T12:36:08.458107+0000","last_clean":"2024-05-27T12:36:08.458107+0000","last_became_active":"2024-05-27T12:36:07.404860+0000","last_became_peered":"2024-05-27T12:36:07.404860+0000","last_unstale":"2024-05-27T12:36:08.458107+0000","last_undegraded":"2024-05-27T12:36:08.458107+0000","last_fullsized":"2024-05-27T12:36:08.458107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503523+0000","last_change":"2024-05-27T12:36:07.406970+0000","last_active":"2024-05-27T12:36:08.503523+0000","last_peered":"2024-05-27T12:36:08.503523+0000","last_clean":"2024-05-27T12:36:08.503523+0000","last_became_active":"2024-05-27T12:36:07.406196+0000","last_became_peered":"2024-05-27T12:36:07.406196+0000","last_unstale":"2024-05-27T12:36:08.503523+0000","last_undegraded":"2024-05-27T12:36:08.503523+0000","last_fullsized":"2024-05-27T12:36:08.503523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500506+0000","last_change":"2024-05-27T12:36:07.412844+0000","last_active":"2024-05-27T12:36:08.500506+0000","last_peered":"2024-05-27T12:36:08.500506+0000","last_clean":"2024-05-27T12:36:08.500506+0000","last_became_active":"2024-05-27T12:36:07.412220+0000","last_became_peered":"2024-05-27T12:36:07.412220+0000","last_unstale":"2024-05-27T12:36:08.500506+0000","last_undegraded":"2024-05-27T12:36:08.500506+0000","last_fullsized":"2024-05-27T12:36:08.500506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:29.723914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500598+0000","last_change":"2024-05-27T12:36:07.401572+0000","last_active":"2024-05-27T12:36:08.500598+0000","last_peered":"2024-05-27T12:36:08.500598+0000","last_clean":"2024-05-27T12:36:08.500598+0000","last_became_active":"2024-05-27T12:36:07.401201+0000","last_became_peered":"2024-05-27T12:36:07.401201+0000","last_unstale":"2024-05-27T12:36:08.500598+0000","last_undegraded":"2024-05-27T12:36:08.500598+0000","last_fullsized":"2024-05-27T12:36:08.500598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:22:03.210565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503410+0000","last_change":"2024-05-27T12:36:07.407396+0000","last_active":"2024-05-27T12:36:08.503410+0000","last_peered":"2024-05-27T12:36:08.503410+0000","last_clean":"2024-05-27T12:36:08.503410+0000","last_became_active":"2024-05-27T12:36:07.407082+0000","last_became_peered":"2024-05-27T12:36:07.407082+0000","last_unstale":"2024-05-27T12:36:08.503410+0000","last_undegraded":"2024-05-27T12:36:08.503410+0000","last_fullsized":"2024-05-27T12:36:08.503410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458466+0000","last_change":"2024-05-27T12:36:07.403712+0000","last_active":"2024-05-27T12:36:08.458466+0000","last_peered":"2024-05-27T12:36:08.458466+0000","last_clean":"2024-05-27T12:36:08.458466+0000","last_became_active":"2024-05-27T12:36:07.402513+0000","last_became_peered":"2024-05-27T12:36:07.402513+0000","last_unstale":"2024-05-27T12:36:08.458466+0000","last_undegraded":"2024-05-27T12:36:08.458466+0000","last_fullsized":"2024-05-27T12:36:08.458466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:47:21.373015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458387+0000","last_change":"2024-05-27T12:36:07.408098+0000","last_active":"2024-05-27T12:36:08.458387+0000","last_peered":"2024-05-27T12:36:08.458387+0000","last_clean":"2024-05-27T12:36:08.458387+0000","last_became_active":"2024-05-27T12:36:07.406544+0000","last_became_peered":"2024-05-27T12:36:07.406544+0000","last_unstale":"2024-05-27T12:36:08.458387+0000","last_undegraded":"2024-05-27T12:36:08.458387+0000","last_fullsized":"2024-05-27T12:36:08.458387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:24.704060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501215+0000","last_change":"2024-05-27T12:36:07.409467+0000","last_active":"2024-05-27T12:36:08.501215+0000","last_peered":"2024-05-27T12:36:08.501215+0000","last_clean":"2024-05-27T12:36:08.501215+0000","last_became_active":"2024-05-27T12:36:07.409210+0000","last_became_peered":"2024-05-27T12:36:07.409210+0000","last_unstale":"2024-05-27T12:36:08.501215+0000","last_undegraded":"2024-05-27T12:36:08.501215+0000","last_fullsized":"2024-05-27T12:36:08.501215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500944+0000","last_change":"2024-05-27T12:36:07.402927+0000","last_active":"2024-05-27T12:36:08.500944+0000","last_peered":"2024-05-27T12:36:08.500944+0000","last_clean":"2024-05-27T12:36:08.500944+0000","last_became_active":"2024-05-27T12:36:07.402642+0000","last_became_peered":"2024-05-27T12:36:07.402642+0000","last_unstale":"2024-05-27T12:36:08.500944+0000","last_undegraded":"2024-05-27T12:36:08.500944+0000","last_fullsized":"2024-05-27T12:36:08.500944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500502+0000","last_change":"2024-05-27T12:36:07.408032+0000","last_active":"2024-05-27T12:36:08.500502+0000","last_peered":"2024-05-27T12:36:08.500502+0000","last_clean":"2024-05-27T12:36:08.500502+0000","last_became_active":"2024-05-27T12:36:07.407755+0000","last_became_peered":"2024-05-27T12:36:07.407755+0000","last_unstale":"2024-05-27T12:36:08.500502+0000","last_undegraded":"2024-05-27T12:36:08.500502+0000","last_fullsized":"2024-05-27T12:36:08.500502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:05.666798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411283+0000","last_change":"2024-05-27T12:36:07.399185+0000","last_active":"2024-05-27T12:36:08.411283+0000","last_peered":"2024-05-27T12:36:08.411283+0000","last_clean":"2024-05-27T12:36:08.411283+0000","last_became_active":"2024-05-27T12:36:07.398851+0000","last_became_peered":"2024-05-27T12:36:07.398851+0000","last_unstale":"2024-05-27T12:36:08.411283+0000","last_undegraded":"2024-05-27T12:36:08.411283+0000","last_fullsized":"2024-05-27T12:36:08.411283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:16.556947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410524+0000","last_change":"2024-05-27T12:36:07.394814+0000","last_active":"2024-05-27T12:36:08.410524+0000","last_peered":"2024-05-27T12:36:08.410524+0000","last_clean":"2024-05-27T12:36:08.410524+0000","last_became_active":"2024-05-27T12:36:07.394471+0000","last_became_peered":"2024-05-27T12:36:07.394471+0000","last_unstale":"2024-05-27T12:36:08.410524+0000","last_undegraded":"2024-05-27T12:36:08.410524+0000","last_fullsized":"2024-05-27T12:36:08.410524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:41.784350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406885+0000","last_change":"2024-05-27T12:36:07.397769+0000","last_active":"2024-05-27T12:36:08.406885+0000","last_peered":"2024-05-27T12:36:08.406885+0000","last_clean":"2024-05-27T12:36:08.406885+0000","last_became_active":"2024-05-27T12:36:07.397423+0000","last_became_peered":"2024-05-27T12:36:07.397423+0000","last_unstale":"2024-05-27T12:36:08.406885+0000","last_undegraded":"2024-05-27T12:36:08.406885+0000","last_fullsized":"2024-05-27T12:36:08.406885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:24.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503104+0000","last_change":"2024-05-27T12:36:07.400408+0000","last_active":"2024-05-27T12:36:08.503104+0000","last_peered":"2024-05-27T12:36:08.503104+0000","last_clean":"2024-05-27T12:36:08.503104+0000","last_became_active":"2024-05-27T12:36:07.400021+0000","last_became_peered":"2024-05-27T12:36:07.400021+0000","last_unstale":"2024-05-27T12:36:08.503104+0000","last_undegraded":"2024-05-27T12:36:08.503104+0000","last_fullsized":"2024-05-27T12:36:08.503104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:46:09.744244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421380+0000","last_change":"2024-05-27T12:36:07.404454+0000","last_active":"2024-05-27T12:36:08.421380+0000","last_peered":"2024-05-27T12:36:08.421380+0000","last_clean":"2024-05-27T12:36:08.421380+0000","last_became_active":"2024-05-27T12:36:07.403996+0000","last_became_peered":"2024-05-27T12:36:07.403996+0000","last_unstale":"2024-05-27T12:36:08.421380+0000","last_undegraded":"2024-05-27T12:36:08.421380+0000","last_fullsized":"2024-05-27T12:36:08.421380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:01.530801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501313+0000","last_change":"2024-05-27T12:36:07.398861+0000","last_active":"2024-05-27T12:36:08.501313+0000","last_peered":"2024-05-27T12:36:08.501313+0000","last_clean":"2024-05-27T12:36:08.501313+0000","last_became_active":"2024-05-27T12:36:07.398499+0000","last_became_peered":"2024-05-27T12:36:07.398499+0000","last_unstale":"2024-05-27T12:36:08.501313+0000","last_undegraded":"2024-05-27T12:36:08.501313+0000","last_fullsized":"2024-05-27T12:36:08.501313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421309+0000","last_change":"2024-05-27T12:36:07.412138+0000","last_active":"2024-05-27T12:36:08.421309+0000","last_peered":"2024-05-27T12:36:08.421309+0000","last_clean":"2024-05-27T12:36:08.421309+0000","last_became_active":"2024-05-27T12:36:07.411322+0000","last_became_peered":"2024-05-27T12:36:07.411322+0000","last_unstale":"2024-05-27T12:36:08.421309+0000","last_undegraded":"2024-05-27T12:36:08.421309+0000","last_fullsized":"2024-05-27T12:36:08.421309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:32:57.241855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503629+0000","last_change":"2024-05-27T12:36:07.404147+0000","last_active":"2024-05-27T12:36:08.503629+0000","last_peered":"2024-05-27T12:36:08.503629+0000","last_clean":"2024-05-27T12:36:08.503629+0000","last_became_active":"2024-05-27T12:36:07.403456+0000","last_became_peered":"2024-05-27T12:36:07.403456+0000","last_unstale":"2024-05-27T12:36:08.503629+0000","last_undegraded":"2024-05-27T12:36:08.503629+0000","last_fullsized":"2024-05-27T12:36:08.503629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:00:23.710267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411653+0000","last_change":"2024-05-27T12:36:07.397416+0000","last_active":"2024-05-27T12:36:08.411653+0000","last_peered":"2024-05-27T12:36:08.411653+0000","last_clean":"2024-05-27T12:36:08.411653+0000","last_became_active":"2024-05-27T12:36:07.394550+0000","last_became_peered":"2024-05-27T12:36:07.394550+0000","last_unstale":"2024-05-27T12:36:08.411653+0000","last_undegraded":"2024-05-27T12:36:08.411653+0000","last_fullsized":"2024-05-27T12:36:08.411653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:48:07.303685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457670+0000","last_change":"2024-05-27T12:36:07.406182+0000","last_active":"2024-05-27T12:36:08.457670+0000","last_peered":"2024-05-27T12:36:08.457670+0000","last_clean":"2024-05-27T12:36:08.457670+0000","last_became_active":"2024-05-27T12:36:07.405895+0000","last_became_peered":"2024-05-27T12:36:07.405895+0000","last_unstale":"2024-05-27T12:36:08.457670+0000","last_undegraded":"2024-05-27T12:36:08.457670+0000","last_fullsized":"2024-05-27T12:36:08.457670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411216+0000","last_change":"2024-05-27T12:36:07.400912+0000","last_active":"2024-05-27T12:36:08.411216+0000","last_peered":"2024-05-27T12:36:08.411216+0000","last_clean":"2024-05-27T12:36:08.411216+0000","last_became_active":"2024-05-27T12:36:07.400267+0000","last_became_peered":"2024-05-27T12:36:07.400267+0000","last_unstale":"2024-05-27T12:36:08.411216+0000","last_undegraded":"2024-05-27T12:36:08.411216+0000","last_fullsized":"2024-05-27T12:36:08.411216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:33:12.319943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457720+0000","last_change":"2024-05-27T12:36:07.407534+0000","last_active":"2024-05-27T12:36:08.457720+0000","last_peered":"2024-05-27T12:36:08.457720+0000","last_clean":"2024-05-27T12:36:08.457720+0000","last_became_active":"2024-05-27T12:36:07.406424+0000","last_became_peered":"2024-05-27T12:36:07.406424+0000","last_unstale":"2024-05-27T12:36:08.457720+0000","last_undegraded":"2024-05-27T12:36:08.457720+0000","last_fullsized":"2024-05-27T12:36:08.457720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406923+0000","last_change":"2024-05-27T12:36:07.400046+0000","last_active":"2024-05-27T12:36:08.406923+0000","last_peered":"2024-05-27T12:36:08.406923+0000","last_clean":"2024-05-27T12:36:08.406923+0000","last_became_active":"2024-05-27T12:36:07.399770+0000","last_became_peered":"2024-05-27T12:36:07.399770+0000","last_unstale":"2024-05-27T12:36:08.406923+0000","last_undegraded":"2024-05-27T12:36:08.406923+0000","last_fullsized":"2024-05-27T12:36:08.406923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:41.784350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411657+0000","last_change":"2024-05-27T12:36:06.399603+0000","last_active":"2024-05-27T12:36:08.411657+0000","last_peered":"2024-05-27T12:36:08.411657+0000","last_clean":"2024-05-27T12:36:08.411657+0000","last_became_active":"2024-05-27T12:36:06.399286+0000","last_became_peered":"2024-05-27T12:36:06.399286+0000","last_unstale":"2024-05-27T12:36:08.411657+0000","last_undegraded":"2024-05-27T12:36:08.411657+0000","last_fullsized":"2024-05-27T12:36:08.411657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411395+0000","last_change":"2024-05-27T12:36:07.390231+0000","last_active":"2024-05-27T12:36:08.411395+0000","last_peered":"2024-05-27T12:36:08.411395+0000","last_clean":"2024-05-27T12:36:08.411395+0000","last_became_active":"2024-05-27T12:36:07.389832+0000","last_became_peered":"2024-05-27T12:36:07.389832+0000","last_unstale":"2024-05-27T12:36:08.411395+0000","last_undegraded":"2024-05-27T12:36:08.411395+0000","last_fullsized":"2024-05-27T12:36:08.411395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:10.994307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406867+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:08.406867+0000","last_peered":"2024-05-27T12:36:08.406867+0000","last_clean":"2024-05-27T12:36:08.406867+0000","last_became_active":"2024-05-27T12:36:06.398355+0000","last_became_peered":"2024-05-27T12:36:06.398355+0000","last_unstale":"2024-05-27T12:36:08.406867+0000","last_undegraded":"2024-05-27T12:36:08.406867+0000","last_fullsized":"2024-05-27T12:36:08.406867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413435+0000","last_change":"2024-05-27T12:36:07.403495+0000","last_active":"2024-05-27T12:36:08.413435+0000","last_peered":"2024-05-27T12:36:08.413435+0000","last_clean":"2024-05-27T12:36:08.413435+0000","last_became_active":"2024-05-27T12:36:07.401246+0000","last_became_peered":"2024-05-27T12:36:07.401246+0000","last_unstale":"2024-05-27T12:36:08.413435+0000","last_undegraded":"2024-05-27T12:36:08.413435+0000","last_fullsized":"2024-05-27T12:36:08.413435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:57:51.876839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500916+0000","last_change":"2024-05-27T12:36:06.393410+0000","last_active":"2024-05-27T12:36:08.500916+0000","last_peered":"2024-05-27T12:36:08.500916+0000","last_clean":"2024-05-27T12:36:08.500916+0000","last_became_active":"2024-05-27T12:36:06.392937+0000","last_became_peered":"2024-05-27T12:36:06.392937+0000","last_unstale":"2024-05-27T12:36:08.500916+0000","last_undegraded":"2024-05-27T12:36:08.500916+0000","last_fullsized":"2024-05-27T12:36:08.500916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:40.771809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421326+0000","last_change":"2024-05-27T12:36:07.411913+0000","last_active":"2024-05-27T12:36:08.421326+0000","last_peered":"2024-05-27T12:36:08.421326+0000","last_clean":"2024-05-27T12:36:08.421326+0000","last_became_active":"2024-05-27T12:36:07.410523+0000","last_became_peered":"2024-05-27T12:36:07.410523+0000","last_unstale":"2024-05-27T12:36:08.421326+0000","last_undegraded":"2024-05-27T12:36:08.421326+0000","last_fullsized":"2024-05-27T12:36:08.421326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:06:56.686640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500613+0000","last_change":"2024-05-27T12:36:06.395813+0000","last_active":"2024-05-27T12:36:08.500613+0000","last_peered":"2024-05-27T12:36:08.500613+0000","last_clean":"2024-05-27T12:36:08.500613+0000","last_became_active":"2024-05-27T12:36:06.392355+0000","last_became_peered":"2024-05-27T12:36:06.392355+0000","last_unstale":"2024-05-27T12:36:08.500613+0000","last_undegraded":"2024-05-27T12:36:08.500613+0000","last_fullsized":"2024-05-27T12:36:08.500613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-29T00:11:10.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457667+0000","last_change":"2024-05-27T12:36:07.410252+0000","last_active":"2024-05-27T12:36:08.457667+0000","last_peered":"2024-05-27T12:36:08.457667+0000","last_clean":"2024-05-27T12:36:08.457667+0000","last_became_active":"2024-05-27T12:36:07.409955+0000","last_became_peered":"2024-05-27T12:36:07.409955+0000","last_unstale":"2024-05-27T12:36:08.457667+0000","last_undegraded":"2024-05-27T12:36:08.457667+0000","last_fullsized":"2024-05-27T12:36:08.457667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:03:27.712175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411888+0000","last_change":"2024-05-27T12:36:06.394640+0000","last_active":"2024-05-27T12:36:08.411888+0000","last_peered":"2024-05-27T12:36:08.411888+0000","last_clean":"2024-05-27T12:36:08.411888+0000","last_became_active":"2024-05-27T12:36:06.394297+0000","last_became_peered":"2024-05-27T12:36:06.394297+0000","last_unstale":"2024-05-27T12:36:08.411888+0000","last_undegraded":"2024-05-27T12:36:08.411888+0000","last_fullsized":"2024-05-27T12:36:08.411888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503216+0000","last_change":"2024-05-27T12:36:07.406862+0000","last_active":"2024-05-27T12:36:08.503216+0000","last_peered":"2024-05-27T12:36:08.503216+0000","last_clean":"2024-05-27T12:36:08.503216+0000","last_became_active":"2024-05-27T12:36:07.404374+0000","last_became_peered":"2024-05-27T12:36:07.404374+0000","last_unstale":"2024-05-27T12:36:08.503216+0000","last_undegraded":"2024-05-27T12:36:08.503216+0000","last_fullsized":"2024-05-27T12:36:08.503216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.410782+0000","last_change":"2024-05-27T12:36:06.394969+0000","last_active":"2024-05-27T12:36:08.410782+0000","last_peered":"2024-05-27T12:36:08.410782+0000","last_clean":"2024-05-27T12:36:08.410782+0000","last_became_active":"2024-05-27T12:36:06.394600+0000","last_became_peered":"2024-05-27T12:36:06.394600+0000","last_unstale":"2024-05-27T12:36:08.410782+0000","last_undegraded":"2024-05-27T12:36:08.410782+0000","last_fullsized":"2024-05-27T12:36:08.410782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:12.023075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412644+0000","last_change":"2024-05-27T12:36:07.389401+0000","last_active":"2024-05-27T12:36:08.412644+0000","last_peered":"2024-05-27T12:36:08.412644+0000","last_clean":"2024-05-27T12:36:08.412644+0000","last_became_active":"2024-05-27T12:36:07.388474+0000","last_became_peered":"2024-05-27T12:36:07.388474+0000","last_unstale":"2024-05-27T12:36:08.412644+0000","last_undegraded":"2024-05-27T12:36:08.412644+0000","last_fullsized":"2024-05-27T12:36:08.412644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:23.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500230+0000","last_change":"2024-05-27T12:36:06.383317+0000","last_active":"2024-05-27T12:36:08.500230+0000","last_peered":"2024-05-27T12:36:08.500230+0000","last_clean":"2024-05-27T12:36:08.500230+0000","last_became_active":"2024-05-27T12:36:06.383006+0000","last_became_peered":"2024-05-27T12:36:06.383006+0000","last_unstale":"2024-05-27T12:36:08.500230+0000","last_undegraded":"2024-05-27T12:36:08.500230+0000","last_fullsized":"2024-05-27T12:36:08.500230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:44:22.857477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420729+0000","last_change":"2024-05-27T12:36:07.415779+0000","last_active":"2024-05-27T12:36:08.420729+0000","last_peered":"2024-05-27T12:36:08.420729+0000","last_clean":"2024-05-27T12:36:08.420729+0000","last_became_active":"2024-05-27T12:36:07.415457+0000","last_became_peered":"2024-05-27T12:36:07.415457+0000","last_unstale":"2024-05-27T12:36:08.420729+0000","last_undegraded":"2024-05-27T12:36:08.420729+0000","last_fullsized":"2024-05-27T12:36:08.420729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:57.100241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.409994+0000","last_change":"2024-05-27T12:36:06.398382+0000","last_active":"2024-05-27T12:36:08.409994+0000","last_peered":"2024-05-27T12:36:08.409994+0000","last_clean":"2024-05-27T12:36:08.409994+0000","last_became_active":"2024-05-27T12:36:06.396324+0000","last_became_peered":"2024-05-27T12:36:06.396324+0000","last_unstale":"2024-05-27T12:36:08.409994+0000","last_undegraded":"2024-05-27T12:36:08.409994+0000","last_fullsized":"2024-05-27T12:36:08.409994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:37:09.492855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504109+0000","last_change":"2024-05-27T12:36:07.402050+0000","last_active":"2024-05-27T12:36:08.504109+0000","last_peered":"2024-05-27T12:36:08.504109+0000","last_clean":"2024-05-27T12:36:08.504109+0000","last_became_active":"2024-05-27T12:36:07.401684+0000","last_became_peered":"2024-05-27T12:36:07.401684+0000","last_unstale":"2024-05-27T12:36:08.504109+0000","last_undegraded":"2024-05-27T12:36:08.504109+0000","last_fullsized":"2024-05-27T12:36:08.504109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:24.704060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500985+0000","last_change":"2024-05-27T12:36:06.400135+0000","last_active":"2024-05-27T12:36:08.500985+0000","last_peered":"2024-05-27T12:36:08.500985+0000","last_clean":"2024-05-27T12:36:08.500985+0000","last_became_active":"2024-05-27T12:36:06.399910+0000","last_became_peered":"2024-05-27T12:36:06.399910+0000","last_unstale":"2024-05-27T12:36:08.500985+0000","last_undegraded":"2024-05-27T12:36:08.500985+0000","last_fullsized":"2024-05-27T12:36:08.500985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:15.544406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501008+0000","last_change":"2024-05-27T12:36:07.413359+0000","last_active":"2024-05-27T12:36:08.501008+0000","last_peered":"2024-05-27T12:36:08.501008+0000","last_clean":"2024-05-27T12:36:08.501008+0000","last_became_active":"2024-05-27T12:36:07.413021+0000","last_became_peered":"2024-05-27T12:36:07.413021+0000","last_unstale":"2024-05-27T12:36:08.501008+0000","last_undegraded":"2024-05-27T12:36:08.501008+0000","last_fullsized":"2024-05-27T12:36:08.501008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:33:15.208968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406992+0000","last_change":"2024-05-27T12:36:06.383845+0000","last_active":"2024-05-27T12:36:08.406992+0000","last_peered":"2024-05-27T12:36:08.406992+0000","last_clean":"2024-05-27T12:36:08.406992+0000","last_became_active":"2024-05-27T12:36:06.383394+0000","last_became_peered":"2024-05-27T12:36:06.383394+0000","last_unstale":"2024-05-27T12:36:08.406992+0000","last_undegraded":"2024-05-27T12:36:08.406992+0000","last_fullsized":"2024-05-27T12:36:08.406992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501478+0000","last_change":"2024-05-27T12:36:07.397487+0000","last_active":"2024-05-27T12:36:08.501478+0000","last_peered":"2024-05-27T12:36:08.501478+0000","last_clean":"2024-05-27T12:36:08.501478+0000","last_became_active":"2024-05-27T12:36:07.397135+0000","last_became_peered":"2024-05-27T12:36:07.397135+0000","last_unstale":"2024-05-27T12:36:08.501478+0000","last_undegraded":"2024-05-27T12:36:08.501478+0000","last_fullsized":"2024-05-27T12:36:08.501478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504250+0000","last_change":"2024-05-27T12:36:06.398488+0000","last_active":"2024-05-27T12:36:08.504250+0000","last_peered":"2024-05-27T12:36:08.504250+0000","last_clean":"2024-05-27T12:36:08.504250+0000","last_became_active":"2024-05-27T12:36:06.397511+0000","last_became_peered":"2024-05-27T12:36:06.397511+0000","last_unstale":"2024-05-27T12:36:08.504250+0000","last_undegraded":"2024-05-27T12:36:08.504250+0000","last_fullsized":"2024-05-27T12:36:08.504250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:56.087700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.503992+0000","last_change":"2024-05-27T12:36:07.407283+0000","last_active":"2024-05-27T12:36:08.503992+0000","last_peered":"2024-05-27T12:36:08.503992+0000","last_clean":"2024-05-27T12:36:08.503992+0000","last_became_active":"2024-05-27T12:36:07.407020+0000","last_became_peered":"2024-05-27T12:36:07.407020+0000","last_unstale":"2024-05-27T12:36:08.503992+0000","last_undegraded":"2024-05-27T12:36:08.503992+0000","last_fullsized":"2024-05-27T12:36:08.503992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.501461+0000","last_change":"2024-05-27T12:36:06.393888+0000","last_active":"2024-05-27T12:36:08.501461+0000","last_peered":"2024-05-27T12:36:08.501461+0000","last_clean":"2024-05-27T12:36:08.501461+0000","last_became_active":"2024-05-27T12:36:06.393567+0000","last_became_peered":"2024-05-27T12:36:06.393567+0000","last_unstale":"2024-05-27T12:36:08.501461+0000","last_undegraded":"2024-05-27T12:36:08.501461+0000","last_fullsized":"2024-05-27T12:36:08.501461+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:03.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420762+0000","last_change":"2024-05-27T12:36:07.418706+0000","last_active":"2024-05-27T12:36:08.420762+0000","last_peered":"2024-05-27T12:36:08.420762+0000","last_clean":"2024-05-27T12:36:08.420762+0000","last_became_active":"2024-05-27T12:36:07.416238+0000","last_became_peered":"2024-05-27T12:36:07.416238+0000","last_unstale":"2024-05-27T12:36:08.420762+0000","last_undegraded":"2024-05-27T12:36:08.420762+0000","last_fullsized":"2024-05-27T12:36:08.420762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:23.778188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412195+0000","last_change":"2024-05-27T12:36:06.396297+0000","last_active":"2024-05-27T12:36:08.412195+0000","last_peered":"2024-05-27T12:36:08.412195+0000","last_clean":"2024-05-27T12:36:08.412195+0000","last_became_active":"2024-05-27T12:36:06.395605+0000","last_became_peered":"2024-05-27T12:36:06.395605+0000","last_unstale":"2024-05-27T12:36:08.412195+0000","last_undegraded":"2024-05-27T12:36:08.412195+0000","last_fullsized":"2024-05-27T12:36:08.412195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500848+0000","last_change":"2024-05-27T12:36:07.400200+0000","last_active":"2024-05-27T12:36:08.500848+0000","last_peered":"2024-05-27T12:36:08.500848+0000","last_clean":"2024-05-27T12:36:08.500848+0000","last_became_active":"2024-05-27T12:36:07.399899+0000","last_became_peered":"2024-05-27T12:36:07.399899+0000","last_unstale":"2024-05-27T12:36:08.500848+0000","last_undegraded":"2024-05-27T12:36:08.500848+0000","last_fullsized":"2024-05-27T12:36:08.500848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:10.190676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420802+0000","last_change":"2024-05-27T12:36:06.391699+0000","last_active":"2024-05-27T12:36:08.420802+0000","last_peered":"2024-05-27T12:36:08.420802+0000","last_clean":"2024-05-27T12:36:08.420802+0000","last_became_active":"2024-05-27T12:36:06.391285+0000","last_became_peered":"2024-05-27T12:36:06.391285+0000","last_unstale":"2024-05-27T12:36:08.420802+0000","last_undegraded":"2024-05-27T12:36:08.420802+0000","last_fullsized":"2024-05-27T12:36:08.420802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458217+0000","last_change":"2024-05-27T12:36:07.406688+0000","last_active":"2024-05-27T12:36:08.458217+0000","last_peered":"2024-05-27T12:36:08.458217+0000","last_clean":"2024-05-27T12:36:08.458217+0000","last_became_active":"2024-05-27T12:36:07.405915+0000","last_became_peered":"2024-05-27T12:36:07.405915+0000","last_unstale":"2024-05-27T12:36:08.458217+0000","last_undegraded":"2024-05-27T12:36:08.458217+0000","last_fullsized":"2024-05-27T12:36:08.458217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:39.937707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500135+0000","last_change":"2024-05-27T12:36:06.399046+0000","last_active":"2024-05-27T12:36:08.500135+0000","last_peered":"2024-05-27T12:36:08.500135+0000","last_clean":"2024-05-27T12:36:08.500135+0000","last_became_active":"2024-05-27T12:36:06.396837+0000","last_became_peered":"2024-05-27T12:36:06.396837+0000","last_unstale":"2024-05-27T12:36:08.500135+0000","last_undegraded":"2024-05-27T12:36:08.500135+0000","last_fullsized":"2024-05-27T12:36:08.500135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:09.981766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.411528+0000","last_change":"2024-05-27T12:36:07.400590+0000","last_active":"2024-05-27T12:36:08.411528+0000","last_peered":"2024-05-27T12:36:08.411528+0000","last_clean":"2024-05-27T12:36:08.411528+0000","last_became_active":"2024-05-27T12:36:07.399770+0000","last_became_peered":"2024-05-27T12:36:07.399770+0000","last_unstale":"2024-05-27T12:36:08.411528+0000","last_undegraded":"2024-05-27T12:36:08.411528+0000","last_fullsized":"2024-05-27T12:36:08.411528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:44:23.870018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500114+0000","last_change":"2024-05-27T12:36:06.397002+0000","last_active":"2024-05-27T12:36:08.500114+0000","last_peered":"2024-05-27T12:36:08.500114+0000","last_clean":"2024-05-27T12:36:08.500114+0000","last_became_active":"2024-05-27T12:36:06.396513+0000","last_became_peered":"2024-05-27T12:36:06.396513+0000","last_unstale":"2024-05-27T12:36:08.500114+0000","last_undegraded":"2024-05-27T12:36:08.500114+0000","last_fullsized":"2024-05-27T12:36:08.500114+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:23.542905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.414036+0000","last_change":"2024-05-27T12:36:07.409445+0000","last_active":"2024-05-27T12:36:08.414036+0000","last_peered":"2024-05-27T12:36:08.414036+0000","last_clean":"2024-05-27T12:36:08.414036+0000","last_became_active":"2024-05-27T12:36:07.409162+0000","last_became_peered":"2024-05-27T12:36:07.409162+0000","last_unstale":"2024-05-27T12:36:08.414036+0000","last_undegraded":"2024-05-27T12:36:08.414036+0000","last_fullsized":"2024-05-27T12:36:08.414036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:13.677115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421487+0000","last_change":"2024-05-27T12:36:06.390945+0000","last_active":"2024-05-27T12:36:08.421487+0000","last_peered":"2024-05-27T12:36:08.421487+0000","last_clean":"2024-05-27T12:36:08.421487+0000","last_became_active":"2024-05-27T12:36:06.390575+0000","last_became_peered":"2024-05-27T12:36:06.390575+0000","last_unstale":"2024-05-27T12:36:08.421487+0000","last_undegraded":"2024-05-27T12:36:08.421487+0000","last_fullsized":"2024-05-27T12:36:08.421487+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504032+0000","last_change":"2024-05-27T12:36:07.413660+0000","last_active":"2024-05-27T12:36:08.504032+0000","last_peered":"2024-05-27T12:36:08.504032+0000","last_clean":"2024-05-27T12:36:08.504032+0000","last_became_active":"2024-05-27T12:36:07.413196+0000","last_became_peered":"2024-05-27T12:36:07.413196+0000","last_unstale":"2024-05-27T12:36:08.504032+0000","last_undegraded":"2024-05-27T12:36:08.504032+0000","last_fullsized":"2024-05-27T12:36:08.504032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:51.089157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413769+0000","last_change":"2024-05-27T12:36:06.396687+0000","last_active":"2024-05-27T12:36:08.413769+0000","last_peered":"2024-05-27T12:36:08.413769+0000","last_clean":"2024-05-27T12:36:08.413769+0000","last_became_active":"2024-05-27T12:36:06.395340+0000","last_became_peered":"2024-05-27T12:36:06.395340+0000","last_unstale":"2024-05-27T12:36:08.413769+0000","last_undegraded":"2024-05-27T12:36:08.413769+0000","last_fullsized":"2024-05-27T12:36:08.413769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:32.986661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421041+0000","last_change":"2024-05-27T12:36:07.417147+0000","last_active":"2024-05-27T12:36:08.421041+0000","last_peered":"2024-05-27T12:36:08.421041+0000","last_clean":"2024-05-27T12:36:08.421041+0000","last_became_active":"2024-05-27T12:36:07.416535+0000","last_became_peered":"2024-05-27T12:36:07.416535+0000","last_unstale":"2024-05-27T12:36:08.421041+0000","last_undegraded":"2024-05-27T12:36:08.421041+0000","last_fullsized":"2024-05-27T12:36:08.421041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:57:51.876839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413707+0000","last_change":"2024-05-27T12:36:06.393791+0000","last_active":"2024-05-27T12:36:08.413707+0000","last_peered":"2024-05-27T12:36:08.413707+0000","last_clean":"2024-05-27T12:36:08.413707+0000","last_became_active":"2024-05-27T12:36:06.391768+0000","last_became_peered":"2024-05-27T12:36:06.391768+0000","last_unstale":"2024-05-27T12:36:08.413707+0000","last_undegraded":"2024-05-27T12:36:08.413707+0000","last_fullsized":"2024-05-27T12:36:08.413707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:17.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500593+0000","last_change":"2024-05-27T12:36:07.399674+0000","last_active":"2024-05-27T12:36:08.500593+0000","last_peered":"2024-05-27T12:36:08.500593+0000","last_clean":"2024-05-27T12:36:08.500593+0000","last_became_active":"2024-05-27T12:36:07.399374+0000","last_became_peered":"2024-05-27T12:36:07.399374+0000","last_unstale":"2024-05-27T12:36:08.500593+0000","last_undegraded":"2024-05-27T12:36:08.500593+0000","last_fullsized":"2024-05-27T12:36:08.500593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:22.574391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.413277+0000","last_change":"2024-05-27T12:36:06.388667+0000","last_active":"2024-05-27T12:36:08.413277+0000","last_peered":"2024-05-27T12:36:08.413277+0000","last_clean":"2024-05-27T12:36:08.413277+0000","last_became_active":"2024-05-27T12:36:06.388369+0000","last_became_peered":"2024-05-27T12:36:06.388369+0000","last_unstale":"2024-05-27T12:36:08.413277+0000","last_undegraded":"2024-05-27T12:36:08.413277+0000","last_fullsized":"2024-05-27T12:36:08.413277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500390+0000","last_change":"2024-05-27T12:36:07.392615+0000","last_active":"2024-05-27T12:36:08.500390+0000","last_peered":"2024-05-27T12:36:08.500390+0000","last_clean":"2024-05-27T12:36:08.500390+0000","last_became_active":"2024-05-27T12:36:07.392259+0000","last_became_peered":"2024-05-27T12:36:07.392259+0000","last_unstale":"2024-05-27T12:36:08.500390+0000","last_undegraded":"2024-05-27T12:36:08.500390+0000","last_fullsized":"2024-05-27T12:36:08.500390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:19.866522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406813+0000","last_change":"2024-05-27T12:36:06.391794+0000","last_active":"2024-05-27T12:36:08.406813+0000","last_peered":"2024-05-27T12:36:08.406813+0000","last_clean":"2024-05-27T12:36:08.406813+0000","last_became_active":"2024-05-27T12:36:06.389977+0000","last_became_peered":"2024-05-27T12:36:06.389977+0000","last_unstale":"2024-05-27T12:36:08.406813+0000","last_undegraded":"2024-05-27T12:36:08.406813+0000","last_fullsized":"2024-05-27T12:36:08.406813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500662+0000","last_change":"2024-05-27T12:36:07.409529+0000","last_active":"2024-05-27T12:36:08.500662+0000","last_peered":"2024-05-27T12:36:08.500662+0000","last_clean":"2024-05-27T12:36:08.500662+0000","last_became_active":"2024-05-27T12:36:07.404579+0000","last_became_peered":"2024-05-27T12:36:07.404579+0000","last_unstale":"2024-05-27T12:36:08.500662+0000","last_undegraded":"2024-05-27T12:36:08.500662+0000","last_fullsized":"2024-05-27T12:36:08.500662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:12.971444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412145+0000","last_change":"2024-05-27T12:36:06.393310+0000","last_active":"2024-05-27T12:36:08.412145+0000","last_peered":"2024-05-27T12:36:08.412145+0000","last_clean":"2024-05-27T12:36:08.412145+0000","last_became_active":"2024-05-27T12:36:06.392014+0000","last_became_peered":"2024-05-27T12:36:06.392014+0000","last_unstale":"2024-05-27T12:36:08.412145+0000","last_undegraded":"2024-05-27T12:36:08.412145+0000","last_fullsized":"2024-05-27T12:36:08.412145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500302+0000","last_change":"2024-05-27T12:36:07.407681+0000","last_active":"2024-05-27T12:36:08.500302+0000","last_peered":"2024-05-27T12:36:08.500302+0000","last_clean":"2024-05-27T12:36:08.500302+0000","last_became_active":"2024-05-27T12:36:07.407404+0000","last_became_peered":"2024-05-27T12:36:07.407404+0000","last_unstale":"2024-05-27T12:36:08.500302+0000","last_undegraded":"2024-05-27T12:36:08.500302+0000","last_fullsized":"2024-05-27T12:36:08.500302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:47:18.345354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412472+0000","last_change":"2024-05-27T12:36:06.393400+0000","last_active":"2024-05-27T12:36:08.412472+0000","last_peered":"2024-05-27T12:36:08.412472+0000","last_clean":"2024-05-27T12:36:08.412472+0000","last_became_active":"2024-05-27T12:36:06.392996+0000","last_became_peered":"2024-05-27T12:36:06.392996+0000","last_unstale":"2024-05-27T12:36:08.412472+0000","last_undegraded":"2024-05-27T12:36:08.412472+0000","last_fullsized":"2024-05-27T12:36:08.412472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:18.853981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458016+0000","last_change":"2024-05-27T12:36:07.408808+0000","last_active":"2024-05-27T12:36:08.458016+0000","last_peered":"2024-05-27T12:36:08.458016+0000","last_clean":"2024-05-27T12:36:08.458016+0000","last_became_active":"2024-05-27T12:36:07.406710+0000","last_became_peered":"2024-05-27T12:36:07.406710+0000","last_unstale":"2024-05-27T12:36:08.458016+0000","last_undegraded":"2024-05-27T12:36:08.458016+0000","last_fullsized":"2024-05-27T12:36:08.458016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:37:14.528854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.502889+0000","last_change":"2024-05-27T12:36:06.396022+0000","last_active":"2024-05-27T12:36:08.502889+0000","last_peered":"2024-05-27T12:36:08.502889+0000","last_clean":"2024-05-27T12:36:08.502889+0000","last_became_active":"2024-05-27T12:36:06.395727+0000","last_became_peered":"2024-05-27T12:36:06.395727+0000","last_unstale":"2024-05-27T12:36:08.502889+0000","last_undegraded":"2024-05-27T12:36:08.502889+0000","last_fullsized":"2024-05-27T12:36:08.502889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T23:48:06.291144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.500734+0000","last_change":"2024-05-27T12:36:07.410480+0000","last_active":"2024-05-27T12:36:08.500734+0000","last_peered":"2024-05-27T12:36:08.500734+0000","last_clean":"2024-05-27T12:36:08.500734+0000","last_became_active":"2024-05-27T12:36:07.406987+0000","last_became_peered":"2024-05-27T12:36:07.406987+0000","last_unstale":"2024-05-27T12:36:08.500734+0000","last_undegraded":"2024-05-27T12:36:08.500734+0000","last_fullsized":"2024-05-27T12:36:08.500734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:39.961835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421551+0000","last_change":"2024-05-27T12:36:06.393032+0000","last_active":"2024-05-27T12:36:08.421551+0000","last_peered":"2024-05-27T12:36:08.421551+0000","last_clean":"2024-05-27T12:36:08.421551+0000","last_became_active":"2024-05-27T12:36:06.392754+0000","last_became_peered":"2024-05-27T12:36:06.392754+0000","last_unstale":"2024-05-27T12:36:08.421551+0000","last_undegraded":"2024-05-27T12:36:08.421551+0000","last_fullsized":"2024-05-27T12:36:08.421551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:48:11.958903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.504031+0000","last_change":"2024-05-27T12:36:07.408023+0000","last_active":"2024-05-27T12:36:08.504031+0000","last_peered":"2024-05-27T12:36:08.504031+0000","last_clean":"2024-05-27T12:36:08.504031+0000","last_became_active":"2024-05-27T12:36:07.407695+0000","last_became_peered":"2024-05-27T12:36:07.407695+0000","last_unstale":"2024-05-27T12:36:08.504031+0000","last_undegraded":"2024-05-27T12:36:08.504031+0000","last_fullsized":"2024-05-27T12:36:08.504031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:39.937707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406962+0000","last_change":"2024-05-27T12:36:06.399546+0000","last_active":"2024-05-27T12:36:08.406962+0000","last_peered":"2024-05-27T12:36:08.406962+0000","last_clean":"2024-05-27T12:36:08.406962+0000","last_became_active":"2024-05-27T12:36:06.399228+0000","last_became_peered":"2024-05-27T12:36:06.399228+0000","last_unstale":"2024-05-27T12:36:08.406962+0000","last_undegraded":"2024-05-27T12:36:08.406962+0000","last_fullsized":"2024-05-27T12:36:08.406962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:21.561850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406980+0000","last_change":"2024-05-27T12:36:07.405026+0000","last_active":"2024-05-27T12:36:08.406980+0000","last_peered":"2024-05-27T12:36:08.406980+0000","last_clean":"2024-05-27T12:36:08.406980+0000","last_became_active":"2024-05-27T12:36:07.404731+0000","last_became_peered":"2024-05-27T12:36:07.404731+0000","last_unstale":"2024-05-27T12:36:08.406980+0000","last_undegraded":"2024-05-27T12:36:08.406980+0000","last_fullsized":"2024-05-27T12:36:08.406980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:56:16.556947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.421180+0000","last_change":"2024-05-27T12:36:06.391239+0000","last_active":"2024-05-27T12:36:08.421180+0000","last_peered":"2024-05-27T12:36:08.421180+0000","last_clean":"2024-05-27T12:36:08.421180+0000","last_became_active":"2024-05-27T12:36:06.390800+0000","last_became_peered":"2024-05-27T12:36:06.390800+0000","last_unstale":"2024-05-27T12:36:08.421180+0000","last_undegraded":"2024-05-27T12:36:08.421180+0000","last_fullsized":"2024-05-27T12:36:08.421180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:38.949294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412409+0000","last_change":"2024-05-27T12:36:07.390309+0000","last_active":"2024-05-27T12:36:08.412409+0000","last_peered":"2024-05-27T12:36:08.412409+0000","last_clean":"2024-05-27T12:36:08.412409+0000","last_became_active":"2024-05-27T12:36:07.389963+0000","last_became_peered":"2024-05-27T12:36:07.389963+0000","last_unstale":"2024-05-27T12:36:08.412409+0000","last_undegraded":"2024-05-27T12:36:08.412409+0000","last_fullsized":"2024-05-27T12:36:08.412409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:24.555446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.407071+0000","last_change":"2024-05-27T12:36:06.389001+0000","last_active":"2024-05-27T12:36:08.407071+0000","last_peered":"2024-05-27T12:36:08.407071+0000","last_clean":"2024-05-27T12:36:08.407071+0000","last_became_active":"2024-05-27T12:36:06.388700+0000","last_became_peered":"2024-05-27T12:36:06.388700+0000","last_unstale":"2024-05-27T12:36:08.407071+0000","last_undegraded":"2024-05-27T12:36:08.407071+0000","last_fullsized":"2024-05-27T12:36:08.407071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.406900+0000","last_change":"2024-05-27T12:36:07.392244+0000","last_active":"2024-05-27T12:36:08.406900+0000","last_peered":"2024-05-27T12:36:08.406900+0000","last_clean":"2024-05-27T12:36:08.406900+0000","last_became_active":"2024-05-27T12:36:07.391822+0000","last_became_peered":"2024-05-27T12:36:07.391822+0000","last_unstale":"2024-05-27T12:36:08.406900+0000","last_undegraded":"2024-05-27T12:36:08.406900+0000","last_fullsized":"2024-05-27T12:36:08.406900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:44:04.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.420901+0000","last_change":"2024-05-27T12:36:06.390271+0000","last_active":"2024-05-27T12:36:08.420901+0000","last_peered":"2024-05-27T12:36:08.420901+0000","last_clean":"2024-05-27T12:36:08.420901+0000","last_became_active":"2024-05-27T12:36:06.389933+0000","last_became_peered":"2024-05-27T12:36:06.389933+0000","last_unstale":"2024-05-27T12:36:08.420901+0000","last_undegraded":"2024-05-27T12:36:08.420901+0000","last_fullsized":"2024-05-27T12:36:08.420901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:52:28.711373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.499988+0000","last_change":"2024-05-27T12:36:07.403635+0000","last_active":"2024-05-27T12:36:08.499988+0000","last_peered":"2024-05-27T12:36:08.499988+0000","last_clean":"2024-05-27T12:36:08.499988+0000","last_became_active":"2024-05-27T12:36:07.403356+0000","last_became_peered":"2024-05-27T12:36:07.403356+0000","last_unstale":"2024-05-27T12:36:08.499988+0000","last_undegraded":"2024-05-27T12:36:08.499988+0000","last_fullsized":"2024-05-27T12:36:08.499988+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:05.585799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412719+0000","last_change":"2024-05-27T12:36:06.389847+0000","last_active":"2024-05-27T12:36:08.412719+0000","last_peered":"2024-05-27T12:36:08.412719+0000","last_clean":"2024-05-27T12:36:08.412719+0000","last_became_active":"2024-05-27T12:36:06.389431+0000","last_became_peered":"2024-05-27T12:36:06.389431+0000","last_unstale":"2024-05-27T12:36:08.412719+0000","last_undegraded":"2024-05-27T12:36:08.412719+0000","last_fullsized":"2024-05-27T12:36:08.412719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:40:04.654257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412201+0000","last_change":"2024-05-27T12:36:07.395868+0000","last_active":"2024-05-27T12:36:08.412201+0000","last_peered":"2024-05-27T12:36:08.412201+0000","last_clean":"2024-05-27T12:36:08.412201+0000","last_became_active":"2024-05-27T12:36:07.395186+0000","last_became_peered":"2024-05-27T12:36:07.395186+0000","last_unstale":"2024-05-27T12:36:08.412201+0000","last_undegraded":"2024-05-27T12:36:08.412201+0000","last_fullsized":"2024-05-27T12:36:08.412201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:17:57.100241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.499993+0000","last_change":"2024-05-27T12:36:06.399243+0000","last_active":"2024-05-27T12:36:08.499993+0000","last_peered":"2024-05-27T12:36:08.499993+0000","last_clean":"2024-05-27T12:36:08.499993+0000","last_became_active":"2024-05-27T12:36:06.397252+0000","last_became_peered":"2024-05-27T12:36:06.397252+0000","last_unstale":"2024-05-27T12:36:08.499993+0000","last_undegraded":"2024-05-27T12:36:08.499993+0000","last_fullsized":"2024-05-27T12:36:08.499993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:33:11.307403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.457795+0000","last_change":"2024-05-27T12:36:07.403646+0000","last_active":"2024-05-27T12:36:08.457795+0000","last_peered":"2024-05-27T12:36:08.457795+0000","last_clean":"2024-05-27T12:36:08.457795+0000","last_became_active":"2024-05-27T12:36:07.403383+0000","last_became_peered":"2024-05-27T12:36:07.403383+0000","last_unstale":"2024-05-27T12:36:08.457795+0000","last_undegraded":"2024-05-27T12:36:08.457795+0000","last_fullsized":"2024-05-27T12:36:08.457795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:06.346932+0000","last_clean_scrub_stamp":"2024-05-27T12:36:06.346932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T22:49:21.153367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.412305+0000","last_change":"2024-05-27T12:36:06.398660+0000","last_active":"2024-05-27T12:36:08.412305+0000","last_peered":"2024-05-27T12:36:08.412305+0000","last_clean":"2024-05-27T12:36:08.412305+0000","last_became_active":"2024-05-27T12:36:06.398205+0000","last_became_peered":"2024-05-27T12:36:06.398205+0000","last_unstale":"2024-05-27T12:36:08.412305+0000","last_undegraded":"2024-05-27T12:36:08.412305+0000","last_fullsized":"2024-05-27T12:36:08.412305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:36:05.334391+0000","last_clean_scrub_stamp":"2024-05-27T12:36:05.334391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:53:23.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-27T12:36:08.458456+0000","last_change":"2024-05-27T12:34:54.417691+0000","last_active":"2024-05-27T12:36:08.458456+0000","last_peered":"2024-05-27T12:36:08.458456+0000","last_clean":"2024-05-27T12:36:08.458456+0000","last_became_active":"2024-05-27T12:34:54.415471+0000","last_became_peered":"2024-05-27T12:34:54.415471+0000","last_unstale":"2024-05-27T12:36:08.458456+0000","last_undegraded":"2024-05-27T12:36:08.458456+0000","last_fullsized":"2024-05-27T12:36:08.458456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T12:34:52.913768+0000","last_clean_scrub_stamp":"2024-05-27T12:34:52.913768+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:46:08.133506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T12:36:17.286 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-27T12:36:17.286 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-27T12:36:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.534+0000 7f9e61400640 1 Processor -- start 2024-05-27T12:36:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.534+0000 7f9e61400640 1 -- start start 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e61400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c072360 0x7f9e5c0727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e61400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9e5c1437d0 0x7f9e5c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e61400640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c06fc30 con 0x7f9e5c143240 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e61400640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c072d00 con 0x7f9e5c1437d0 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e61400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c0cd1a0 con 0x7f9e5c072360 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e5b200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9e5c1437d0 0x7f9e5c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e5b200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9e5c1437d0 0x7f9e5c143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46312/0 (socket says 172.21.15.94:46312) 2024-05-27T12:36:17.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e5b200640 1 -- 172.21.15.94:0/1757210962 learned_addr learned my addr 172.21.15.94:0/1757210962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e5a800640 1 --1- 172.21.15.94:0/1757210962 >> v1:172.21.15.94:6789/0 conn(0x7f9e5c143240 0x7f9e5c06f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36610/0 (socket says 172.21.15.94:36610) 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.536+0000 7f9e59e00640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c072360 0x7f9e5c0727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.537+0000 7f9e5b200640 1 -- 172.21.15.94:0/1757210962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c072360 msgr2=0x7f9e5c0727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.537+0000 7f9e5b200640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c072360 0x7f9e5c0727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.537+0000 7f9e5b200640 1 -- 172.21.15.94:0/1757210962 >> v1:172.21.15.94:6789/0 conn(0x7f9e5c143240 legacy=0x7f9e5c06f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.537+0000 7f9e5b200640 1 -- 172.21.15.94:0/1757210962 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c0cd480 con 0x7f9e5c1437d0 2024-05-27T12:36:17.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.537+0000 7f9e59400640 1 -- 172.21.15.94:0/1757210962 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016867473 0 0) 0x7f9e5c06fc30 con 0x7f9e5c143240 2024-05-27T12:36:17.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.538+0000 7f9e5b200640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c143c60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009e50 tx=0x7f9e5402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe112f023c14fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.538+0000 7f9e59400640 1 -- 172.21.15.94:0/1757210962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54036070 con 0x7f9e5c1437d0 2024-05-27T12:36:17.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.538+0000 7f9e59400640 1 -- 172.21.15.94:0/1757210962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e54031600 con 0x7f9e5c1437d0 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.538+0000 7f9e59400640 1 -- 172.21.15.94:0/1757210962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540319d0 con 0x7f9e5c1437d0 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 -- 172.21.15.94:0/1757210962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 msgr2=0x7f9e5c143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c143c60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009e50 tx=0x7f9e5402a680 comp rx=0 tx=0).stop 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 -- 172.21.15.94:0/1757210962 shutdown_connections 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c143c60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 --2- 172.21.15.94:0/1757210962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c072360 0x7f9e5c0727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 -- 172.21.15.94:0/1757210962 >> 172.21.15.94:0/1757210962 conn(0x7f9e5c0b61d0 msgr2=0x7f9e5c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 -- 172.21.15.94:0/1757210962 shutdown_connections 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.539+0000 7f9e61400640 1 -- 172.21.15.94:0/1757210962 wait complete. 2024-05-27T12:36:17.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.540+0000 7f9e61400640 1 Processor -- start 2024-05-27T12:36:17.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.541+0000 7f9e61400640 1 -- start start 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e5c072360 0x7f9e5c0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c143240 0x7f9e5c14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c0cfb30 con 0x7f9e5c1437d0 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c06f8c0 con 0x7f9e5c072360 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e61400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c0cdc80 con 0x7f9e5c143240 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46314/0 (socket says 172.21.15.94:46314) 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 -- 172.21.15.94:0/1707240372 learned_addr learned my addr 172.21.15.94:0/1707240372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5a800640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e5c072360 0x7f9e5c0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e59e00640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c143240 0x7f9e5c14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 -- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c143240 msgr2=0x7f9e5c14cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c143240 0x7f9e5c14cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 -- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e5c072360 msgr2=0x7f9e5c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e5c072360 0x7f9e5c0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.542+0000 7f9e5b200640 1 -- 172.21.15.94:0/1707240372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c14bb60 con 0x7f9e5c1437d0 2024-05-27T12:36:17.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e5b200640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e54034040 tx=0x7f9e54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54041020 con 0x7f9e5c1437d0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e54007780 con 0x7f9e5c1437d0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54036040 con 0x7f9e5c1437d0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c14be40 con 0x7f9e5c1437d0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.543+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c14c400 con 0x7f9e5c1437d0 2024-05-27T12:36:17.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.545+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c068b50 con 0x7f9e5c1437d0 2024-05-27T12:36:17.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.546+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e54044020 con 0x7f9e5c1437d0 2024-05-27T12:36:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.547+0000 7f9e52a00640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.547+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9e540ac8c0 con 0x7f9e5c1437d0 2024-05-27T12:36:17.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.547+0000 7f9e5a800640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.548+0000 7f9e5a800640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000f60 tx=0x7f9e4c007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.551+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e5404b030 con 0x7f9e5c1437d0 2024-05-27T12:36:17.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.836+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9e5c06cf30 con 0x7f9e5c1437d0 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.841+0000 7f9e52a00640 1 -- 172.21.15.94:0/1707240372 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f9e5407ac60 con 0x7f9e5c1437d0 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQC7fVRm82KsBhAApDuhvkqez8bDlFwJfJwTsw== 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-05-27T12:36:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-27T12:36:17.841 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-05-27T12:36:17.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 msgr2=0x7f9e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000f60 tx=0x7f9e4c007b50 comp rx=0 tx=0).stop 2024-05-27T12:36:17.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 msgr2=0x7f9e5c0ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:17.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e54034040 tx=0x7f9e54004a40 comp rx=0 tx=0).stop 2024-05-27T12:36:17.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 shutdown_connections 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9e3c06c8f0 0x7f9e3c06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5c1437d0 0x7f9e5c0ab3c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9e5c143240 0x7f9e5c14cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 --2- 172.21.15.94:0/1707240372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e5c072360 0x7f9e5c0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.843+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 >> 172.21.15.94:0/1707240372 conn(0x7f9e5c0b61d0 msgr2=0x7f9e5c06ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.844+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 shutdown_connections 2024-05-27T12:36:17.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:17.844+0000 7f9e61400640 1 -- 172.21.15.94:0/1707240372 wait complete. 2024-05-27T12:36:17.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T12:36:18.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.058+0000 7f396d800640 1 Processor -- start 2024-05-27T12:36:18.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.058+0000 7f396d800640 1 -- start start 2024-05-27T12:36:18.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f396d800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39681460d0 0x7f3968160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f396d800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3968071d60 0x7f39680721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f396d800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39680ceae0 con 0x7f3968145cf0 2024-05-27T12:36:18.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f396d800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f39681602c0 con 0x7f39681460d0 2024-05-27T12:36:18.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f396d800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f39681605a0 con 0x7f3968071d60 2024-05-27T12:36:18.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f395f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39681460d0 0x7f3968160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f395f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f39681460d0 0x7f3968160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46318/0 (socket says 172.21.15.94:46318) 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.060+0000 7f395f400640 1 -- 172.21.15.94:0/756566530 learned_addr learned my addr 172.21.15.94:0/756566530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.061+0000 7f396ce00640 1 --2- 172.21.15.94:0/756566530 >> v2:172.21.15.97:3300/0 conn(0x7f3968071d60 0x7f39680721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.061+0000 7f395f400640 1 -- 172.21.15.94:0/756566530 >> v2:172.21.15.97:3300/0 conn(0x7f3968071d60 msgr2=0x7f39680721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.061+0000 7f395f400640 1 --2- 172.21.15.94:0/756566530 >> v2:172.21.15.97:3300/0 conn(0x7f3968071d60 0x7f39680721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.061+0000 7f395f400640 1 -- 172.21.15.94:0/756566530 >> v1:172.21.15.97:6789/0 conn(0x7f3968145cf0 legacy=0x7f39680ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.061+0000 7f395f400640 1 -- 172.21.15.94:0/756566530 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3968160880 con 0x7f39681460d0 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.062+0000 7f395f400640 1 --2- 172.21.15.94:0/756566530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39681460d0 0x7f3968160da0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39680b00a0 tx=0x7f395002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540cd28bdb73dd48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.062+0000 7f3967400640 1 -- 172.21.15.94:0/756566530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950037070 con 0x7f39681460d0 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.062+0000 7f3967400640 1 -- 172.21.15.94:0/756566530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3950032440 con 0x7f39681460d0 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.062+0000 7f3967400640 1 -- 172.21.15.94:0/756566530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950032810 con 0x7f39681460d0 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 -- 172.21.15.94:0/756566530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39681460d0 msgr2=0x7f3968160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 --2- 172.21.15.94:0/756566530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39681460d0 0x7f3968160da0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f39680b00a0 tx=0x7f395002a230 comp rx=0 tx=0).stop 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 -- 172.21.15.94:0/756566530 shutdown_connections 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 --2- 172.21.15.94:0/756566530 >> v2:172.21.15.97:3300/0 conn(0x7f3968071d60 0x7f39680721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 --2- 172.21.15.94:0/756566530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39681460d0 0x7f3968160da0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 -- 172.21.15.94:0/756566530 >> 172.21.15.94:0/756566530 conn(0x7f39680b2160 msgr2=0x7f39680b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 -- 172.21.15.94:0/756566530 shutdown_connections 2024-05-27T12:36:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.063+0000 7f396d800640 1 -- 172.21.15.94:0/756566530 wait complete. 2024-05-27T12:36:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.065+0000 7f396d800640 1 Processor -- start 2024-05-27T12:36:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.065+0000 7f396d800640 1 -- start start 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968071d60 0x7f396815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39681460d0 0x7f39680c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f396806eb00 con 0x7f3968145cf0 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39680cedc0 con 0x7f3968071d60 2024-05-27T12:36:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f396d800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f396806ec80 con 0x7f39681460d0 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f395f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f395f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46324/0 (socket says 172.21.15.94:46324) 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f395f400640 1 -- 172.21.15.94:0/1899667616 learned_addr learned my addr 172.21.15.94:0/1899667616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.066+0000 7f3967e00640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968071d60 0x7f396815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 -- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39681460d0 msgr2=0x7f39680c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39681460d0 0x7f39680c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 -- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968071d60 msgr2=0x7f396815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968071d60 0x7f396815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 -- 172.21.15.94:0/1899667616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680af1a0 con 0x7f3968145cf0 2024-05-27T12:36:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f395f400640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3950000c00 tx=0x7f39500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950041020 con 0x7f3968145cf0 2024-05-27T12:36:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39500306c0 con 0x7f3968145cf0 2024-05-27T12:36:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950037040 con 0x7f3968145cf0 2024-05-27T12:36:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.067+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39680af480 con 0x7f3968145cf0 2024-05-27T12:36:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.068+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396815d410 con 0x7f3968145cf0 2024-05-27T12:36:18.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.069+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3968075460 con 0x7f3968145cf0 2024-05-27T12:36:18.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.070+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3950044020 con 0x7f3968145cf0 2024-05-27T12:36:18.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.070+0000 7f3964c00640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 0x7f394406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.071+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f39500ac3c0 con 0x7f3968145cf0 2024-05-27T12:36:18.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.071+0000 7f3967e00640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 0x7f394406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.072+0000 7f3967e00640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 0x7f394406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.075+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f395004b030 con 0x7f3968145cf0 2024-05-27T12:36:18.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.314+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3968072e00 con 0x7f3968145cf0 2024-05-27T12:36:18.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.315+0000 7f3964c00640 1 -- 172.21.15.94:0/1899667616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f395007a7e0 con 0x7f3968145cf0 2024-05-27T12:36:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:36:18.315 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1231254464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1231254464},{"type":"v1","addr":"172.21.15.94:6835","nonce":1231254464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/750013346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":750013346},{"type":"v1","addr":"172.21.15.97:6837","nonce":750013346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4723,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/956232445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":956232445},{"type":"v1","addr":"172.21.15.97:6835","nonce":956232445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T12:36:06.826950+0000","modified":"2024-05-27T12:36:09.894453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4971},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4971":{"gid":4971,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.94:6834/2304976623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2304976623},{"type":"v1","addr":"172.21.15.94:6834","nonce":2304976623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4971,"qdb_cluster":[4971]},"id":2}]} 2024-05-27T12:36:18.315 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-05-27T12:36:18.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 msgr2=0x7f394406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 0x7f394406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958009040 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 msgr2=0x7f39680ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3950000c00 tx=0x7f39500046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 shutdown_connections 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f394406c8f0 0x7f394406edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39681460d0 0x7f39680c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3968145cf0 0x7f39680ae6b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 --2- 172.21.15.94:0/1899667616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968071d60 0x7f396815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 >> 172.21.15.94:0/1899667616 conn(0x7f39680b2160 msgr2=0x7f39680cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 shutdown_connections 2024-05-27T12:36:18.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.318+0000 7f396d800640 1 -- 172.21.15.94:0/1899667616 wait complete. 2024-05-27T12:36:18.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T12:36:06.826950+0000', 'modified': '2024-05-27T12:36:09.894453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4971}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4971': {'gid': 4971, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.94:6834/2304976623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6832', 'nonce': 2304976623}, {'type': 'v1', 'addr': '172.21.15.94:6834', 'nonce': 2304976623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4971, 'qdb_cluster': [4971]} 2024-05-27T12:36:18.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T12:36:18.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-27T12:36:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.951+0000 7f2ae3200640 1 Processor -- start 2024-05-27T12:36:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.952+0000 7f2ae3200640 1 -- start start 2024-05-27T12:36:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.953+0000 7f2ae3200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae4071d40 0x7f2ae40721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.953+0000 7f2ae3200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae4161000 con 0x7f2ae4145df0 2024-05-27T12:36:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.953+0000 7f2ae3200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae4160d20 con 0x7f2ae4160140 2024-05-27T12:36:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.953+0000 7f2ae3200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae40726f0 con 0x7f2ae4071d40 2024-05-27T12:36:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae4071d40 0x7f2ae40721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2ae4071d40 0x7f2ae40721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46330/0 (socket says 172.21.15.94:46330) 2024-05-27T12:36:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 -- 172.21.15.94:0/521328346 learned_addr learned my addr 172.21.15.94:0/521328346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1e00640 1 --1- 172.21.15.94:0/521328346 >> v1:172.21.15.94:6789/0 conn(0x7f2ae4145df0 0x7f2ae40cda10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36622/0 (socket says 172.21.15.94:36622) 2024-05-27T12:36:18.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 -- 172.21.15.94:0/521328346 >> v1:172.21.15.97:6789/0 conn(0x7f2ae4160140 legacy=0x7f2ae4160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 -- 172.21.15.94:0/521328346 >> v1:172.21.15.94:6789/0 conn(0x7f2ae4145df0 legacy=0x7f2ae40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae1400640 1 -- 172.21.15.94:0/521328346 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae40ccf30 con 0x7f2ae4071d40 2024-05-27T12:36:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.954+0000 7f2ae0a00640 1 -- 172.21.15.94:0/521328346 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538885715 0 0) 0x7f2ae4161000 con 0x7f2ae4145df0 2024-05-27T12:36:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.955+0000 7f2ae1400640 1 --2- 172.21.15.94:0/521328346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4071d40 0x7f2ae40721b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009e50 tx=0x7f2ad402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1979147452b0d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae0a00640 1 -- 172.21.15.94:0/521328346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4036070 con 0x7f2ae4071d40 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae0a00640 1 -- 172.21.15.94:0/521328346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031600 con 0x7f2ae4071d40 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae0a00640 1 -- 172.21.15.94:0/521328346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40319d0 con 0x7f2ae4071d40 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae3200640 1 -- 172.21.15.94:0/521328346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4071d40 msgr2=0x7f2ae40721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae3200640 1 --2- 172.21.15.94:0/521328346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4071d40 0x7f2ae40721b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009e50 tx=0x7f2ad402a6d0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae3200640 1 -- 172.21.15.94:0/521328346 shutdown_connections 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae3200640 1 --2- 172.21.15.94:0/521328346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4071d40 0x7f2ae40721b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.956+0000 7f2ae3200640 1 -- 172.21.15.94:0/521328346 >> 172.21.15.94:0/521328346 conn(0x7f2ae406b930 msgr2=0x7f2ae406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.957+0000 7f2ae3200640 1 -- 172.21.15.94:0/521328346 shutdown_connections 2024-05-27T12:36:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.957+0000 7f2ae3200640 1 -- 172.21.15.94:0/521328346 wait complete. 2024-05-27T12:36:18.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.958+0000 7f2ae3200640 1 Processor -- start 2024-05-27T12:36:18.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.958+0000 7f2ae3200640 1 -- start start 2024-05-27T12:36:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ae4071d40 0x7f2ae413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae4160140 0x7f2ae406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41461d0 con 0x7f2ae4145df0 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40cd4f0 con 0x7f2ae4160140 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.959+0000 7f2ae3200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae40b3460 con 0x7f2ae4071d40 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46346/0 (socket says 172.21.15.94:46346) 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 -- 172.21.15.94:0/2347475450 learned_addr learned my addr 172.21.15.94:0/2347475450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 -- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ae4071d40 msgr2=0x7f2ae413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ae4071d40 0x7f2ae413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 -- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae4160140 msgr2=0x7f2ae406e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae2800640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae4160140 0x7f2ae406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae4160140 0x7f2ae406e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 -- 172.21.15.94:0/2347475450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae413f140 con 0x7f2ae4145df0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.960+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4034040 tx=0x7f2ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.961+0000 7f2ada000640 1 -- 172.21.15.94:0/2347475450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4041030 con 0x7f2ae4145df0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.961+0000 7f2ada000640 1 -- 172.21.15.94:0/2347475450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031400 con 0x7f2ae4145df0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.961+0000 7f2ada000640 1 -- 172.21.15.94:0/2347475450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4036040 con 0x7f2ae4145df0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.961+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae406df00 con 0x7f2ae4145df0 2024-05-27T12:36:18.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.961+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae406e4c0 con 0x7f2ae4145df0 2024-05-27T12:36:18.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.964+0000 7f2ada000640 1 -- 172.21.15.94:0/2347475450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ad4044070 con 0x7f2ae4145df0 2024-05-27T12:36:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.965+0000 7f2ada000640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.965+0000 7f2ada000640 1 -- 172.21.15.94:0/2347475450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2ad40ac700 con 0x7f2ae4145df0 2024-05-27T12:36:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.965+0000 7f2aeb96d740 1 Processor -- start 2024-05-27T12:36:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.966+0000 7f2aeb96d740 1 -- start start 2024-05-27T12:36:18.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.966+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.966+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d365490 0x55843d368c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d3855d0 0x55843d385a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55843d2a2b10 con 0x55843d369150 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55843d2492b0 con 0x55843d3855d0 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x55843d29ead0 con 0x55843d365490 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46348/0 (socket says 172.21.15.94:46348) 2024-05-27T12:36:18.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 -- 172.21.15.94:0/2681054904 learned_addr learned my addr 172.21.15.94:0/2681054904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2acc007fd0 tx=0x7f2acc007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 -- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d365490 msgr2=0x55843d368c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d365490 0x55843d368c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 -- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d3855d0 msgr2=0x55843d385a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d3855d0 0x55843d385a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 -- 172.21.15.94:0/2681054904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843cfb52d0 con 0x55843d369150 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.967+0000 7f2ae1400640 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4082a40 tx=0x7f2ad4009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d634012907a3bd0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2ad2600640 1 -- 172.21.15.94:0/2681054904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40a9070 con 0x55843d369150 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2ad2600640 1 -- 172.21.15.94:0/2681054904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad407a390 con 0x55843d369150 2024-05-27T12:36:18.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2ad2600640 1 -- 172.21.15.94:0/2681054904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad407e740 con 0x55843d369150 2024-05-27T12:36:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 -- 172.21.15.94:0/2681054904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 msgr2=0x55843d385090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4082a40 tx=0x7f2ad4009610 comp rx=0 tx=0).stop 2024-05-27T12:36:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 -- 172.21.15.94:0/2681054904 shutdown_connections 2024-05-27T12:36:18.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d3855d0 0x55843d385a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d385090 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.968+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/2681054904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d365490 0x55843d368c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.969+0000 7f2aeb96d740 1 -- 172.21.15.94:0/2681054904 >> 172.21.15.94:0/2681054904 conn(0x55843d33b470 msgr2=0x55843d33c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.969+0000 7f2aeb96d740 1 -- 172.21.15.94:0/2681054904 shutdown_connections 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.969+0000 7f2aeb96d740 1 -- 172.21.15.94:0/2681054904 wait complete. 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.969+0000 7f2aeb96d740 1 -- start start 2024-05-27T12:36:18.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.969+0000 7f2aeb96d740 1 Processor -- start 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.970+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d369150 0x55843d37a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.970+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.970+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d37b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.970+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55843d2a2b10 con 0x55843d3855d0 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55843d25c570 con 0x55843d360ef0 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x55843d3392a0 con 0x55843d369150 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2aeb96d740 10 client.0 _ll_register_callbacks cb 0x55843d22afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46356/0 (socket says 172.21.15.94:46356) 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 -- 172.21.15.94:0/3303258885 learned_addr learned my addr 172.21.15.94:0/3303258885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d369150 0x55843d37a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae2800640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d37b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 -- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d369150 msgr2=0x55843d37a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d369150 0x55843d37a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 -- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 msgr2=0x55843d37b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d37b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.971+0000 7f2ae1400640 1 -- 172.21.15.94:0/3303258885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843d07f6b0 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ae1400640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4092040 tx=0x7f2ad402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9cf0bef290bce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40a9040 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2ad4078690 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad407a390 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2ad407e740 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2aeb96d740 10 client.5193 fetch_fsmap learned FSMap version 17 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55843d39cd90 con 0x55843d3855d0 2024-05-27T12:36:18.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad407a390 con 0x55843d3855d0 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f2ad4092b70 con 0x55843d3855d0 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.972+0000 7f2aeb96d740 10 client.5193 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2aeb96d740 10 client.5193 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2aeb96d740 20 client.5193 populate_metadata read hostname 'smithi094' 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d39cd90 0x55843d3a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2aeb96d740 4 client.5193 mds_command: new command op to 4971 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 --> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843d2b0530 con 0x55843d39cd90 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d39cd90 0x55843d3a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d39cd90 0x55843d3a12f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262e9b5d62fa2a58 server_cookie=e572e91ca084b53b in_seq=0 out_seq=0 2024-05-27T12:36:18.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.973+0000 7f2ac6a00640 10 client.5193 ms_handle_connect on v2:172.21.15.94:6832/2304976623 2024-05-27T12:36:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.974+0000 7f2ac6a00640 1 -- 172.21.15.94:0/3303258885 <== mds.0 v2:172.21.15.94:6832/2304976623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55843d2b0530 con 0x55843d39cd90 2024-05-27T12:36:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.974+0000 7f2ac6a00640 10 client.5193 handle_command_reply: tid=0 2024-05-27T12:36:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.974+0000 7f2aeb96d740 1 client.5193 shutdown 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 msgr2=0x55843d37adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4092040 tx=0x7f2ad402a6d0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 shutdown_connections 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d39cd90 0x55843d3a12f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d37b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d3855d0 0x55843d37adb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2ac6a00640 0 client.5193 ms_handle_reset on v2:172.21.15.94:6832/2304976623 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/3303258885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d369150 0x55843d37a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 >> 172.21.15.94:0/3303258885 conn(0x55843d365490 msgr2=0x55843d3381b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 shutdown_connections 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 1 -- 172.21.15.94:0/3303258885 wait complete. 2024-05-27T12:36:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.975+0000 7f2aeb96d740 20 client.5193 trim_cache size 0 max 16384 2024-05-27T12:36:18.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.995+0000 7f2aeb96d740 1 Processor -- start 2024-05-27T12:36:18.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.995+0000 7f2aeb96d740 1 -- start start 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d392520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d3a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ffbb0 con 0x55843d369150 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41fef40 con 0x55843d360ef0 2024-05-27T12:36:18.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae41ff560 con 0x55843d3855d0 2024-05-27T12:36:18.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46368/0 (socket says 172.21.15.94:46368) 2024-05-27T12:36:18.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 -- 172.21.15.94:0/105631795 learned_addr learned my addr 172.21.15.94:0/105631795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:18.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d392520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:18.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 -- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 msgr2=0x55843d3a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d3a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 -- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 msgr2=0x55843d392520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d392520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.996+0000 7f2ae1400640 1 -- 172.21.15.94:0/105631795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843d33abe0 con 0x55843d369150 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.997+0000 7f2ae1400640 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000f80 tx=0x7f2ad40aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d15e4393c1dd505 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.997+0000 7f2ad1c00640 1 -- 172.21.15.94:0/105631795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40315d0 con 0x55843d369150 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.997+0000 7f2ad1c00640 1 -- 172.21.15.94:0/105631795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad407a390 con 0x55843d369150 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.997+0000 7f2ad1c00640 1 -- 172.21.15.94:0/105631795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4092b70 con 0x55843d369150 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- 172.21.15.94:0/105631795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 msgr2=0x55843d3a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000f80 tx=0x7f2ad40aa000 comp rx=0 tx=0).stop 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- 172.21.15.94:0/105631795 shutdown_connections 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d3a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d369150 0x55843d3a6550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/105631795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d360ef0 0x55843d392520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- 172.21.15.94:0/105631795 >> 172.21.15.94:0/105631795 conn(0x55843d365490 msgr2=0x55843d364aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- 172.21.15.94:0/105631795 shutdown_connections 2024-05-27T12:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- 172.21.15.94:0/105631795 wait complete. 2024-05-27T12:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.998+0000 7f2aeb96d740 1 -- start start 2024-05-27T12:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:18.999+0000 7f2aeb96d740 1 Processor -- start 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d369150 0x55843d395840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d395d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55843d2a2b10 con 0x55843d360ef0 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55843d25c570 con 0x55843d369150 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x55843d39d490 con 0x55843d3855d0 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2aeb96d740 10 client.0 _ll_register_callbacks cb 0x7f2ad4079f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46380/0 (socket says 172.21.15.94:46380) 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 -- 172.21.15.94:0/595639414 learned_addr learned my addr 172.21.15.94:0/595639414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 -- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 msgr2=0x55843d395d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d395d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 -- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d369150 msgr2=0x55843d395840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d369150 0x55843d395840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.000+0000 7f2ae1e00640 1 -- 172.21.15.94:0/595639414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843d380220 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ae1e00640 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0011e0 tx=0x7f2acc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb618012eed2d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc026020 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2acc00c930 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad407a390 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2acc030050 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 10 client.5199 fetch_fsmap learned FSMap version 17 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55843d3807e0 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad407a390 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f2acc033020 con 0x55843d360ef0 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 10 client.5199 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T12:36:19.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 10 client.5199 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 20 client.5199 populate_metadata read hostname 'smithi094' 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d33ee20 0x55843d387190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 4 client.5199 mds_command: new command op to 4971 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.001+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 --> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] -- command(tid 0: {"prefix": "status"}) -- 0x55843d2aa3a0 con 0x55843d33ee20 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2ae1400640 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d33ee20 0x55843d387190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2ae1400640 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d33ee20 0x55843d387190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746c8a8032418a50 server_cookie=c7b7e4c7553d53df in_seq=0 out_seq=0 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2ac6a00640 10 client.5199 ms_handle_connect on v2:172.21.15.94:6832/2304976623 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2ac6a00640 1 -- 172.21.15.94:0/595639414 <== mds.0 v2:172.21.15.94:6832/2304976623 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55843d2aa3a0 con 0x55843d33ee20 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2ac6a00640 10 client.5199 handle_command_reply: tid=0 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.002+0000 7f2aeb96d740 1 client.5199 shutdown 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 msgr2=0x55843d36af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0011e0 tx=0x7f2acc007780 comp rx=0 tx=0).stop 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 shutdown_connections 2024-05-27T12:36:19.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:6832/2304976623,v1:172.21.15.94:6834/2304976623] conn(0x55843d33ee20 0x55843d387190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55843d369150 0x55843d395840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55843d3855d0 0x55843d395d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 --2- 172.21.15.94:0/595639414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55843d360ef0 0x55843d36af90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 >> 172.21.15.94:0/595639414 conn(0x55843d365490 msgr2=0x55843d340170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2ac6a00640 0 client.5199 ms_handle_reset on v2:172.21.15.94:6832/2304976623 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 shutdown_connections 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 1 -- 172.21.15.94:0/595639414 wait complete. 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.003+0000 7f2aeb96d740 20 client.5199 trim_cache size 0 max 16384 2024-05-27T12:36:19.006 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4971, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 11.612918495000001, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 17, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 33, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 33, 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 20.459475779999998 2024-05-27T12:36:19.007 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:36:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 msgr2=0x7f2ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2acc007fd0 tx=0x7f2acc007920 comp rx=0 tx=0).stop 2024-05-27T12:36:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 msgr2=0x7f2ae413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4034040 tx=0x7f2ad4004a40 comp rx=0 tx=0).stop 2024-05-27T12:36:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 shutdown_connections 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae4160140 0x7f2ae406e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae4145df0 0x7f2ae413e760 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 --2- 172.21.15.94:0/2347475450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ae4071d40 0x7f2ae413fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 >> 172.21.15.94:0/2347475450 conn(0x7f2ae406b930 msgr2=0x7f2ae406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 shutdown_connections 2024-05-27T12:36:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.004+0000 7f2ae3200640 1 -- 172.21.15.94:0/2347475450 wait complete. 2024-05-27T12:36:19.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T12:36:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.213+0000 7fc275800640 1 Processor -- start 2024-05-27T12:36:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.213+0000 7fc275800640 1 -- start start 2024-05-27T12:36:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.214+0000 7fc275800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 0x7fc270072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.214+0000 7fc275800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2700cf040 0x7fc2700cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.214+0000 7fc275800640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2700ce200 con 0x7fc270144bb0 2024-05-27T12:36:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.214+0000 7fc275800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc270144f90 con 0x7fc2700cf040 2024-05-27T12:36:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.214+0000 7fc275800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc2700728c0 con 0x7fc270071f20 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc26ea00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc270144bb0 0x7fc2700cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:36634/0 (socket says 172.21.15.94:36634) 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc26ea00640 1 -- 172.21.15.94:0/1760357149 learned_addr learned my addr 172.21.15.94:0/1760357149 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc26f400640 1 --2- 172.21.15.94:0/1760357149 >> v2:172.21.15.97:3300/0 conn(0x7fc2700cf040 0x7fc2700cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc267e00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 0x7fc270072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551231084 0 0) 0x7fc2700ce200 con 0x7fc270144bb0 2024-05-27T12:36:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.215+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc254003520 con 0x7fc270144bb0 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 381196744 0 0) 0x7fc254003520 con 0x7fc270144bb0 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 msgr2=0x7fc270072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 0x7fc270072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 >> v2:172.21.15.97:3300/0 conn(0x7fc2700cf040 msgr2=0x7fc2700cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 --2- 172.21.15.94:0/1760357149 >> v2:172.21.15.97:3300/0 conn(0x7fc2700cf040 0x7fc2700cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700cdc40 con 0x7fc270144bb0 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fc260001670 con 0x7fc270144bb0 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 >> v1:172.21.15.94:6789/0 conn(0x7fc270144bb0 legacy=0x7fc2700cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254003a90 0x7fc254003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254004800 0x7fc254004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 0x7fc2540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc270144f90 con 0x7fc254003a90 2024-05-27T12:36:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc25400dbd0 con 0x7fc254004800 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc25400dd80 con 0x7fc270071f20 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26ea00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254003a90 0x7fc254003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc267e00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254004800 0x7fc254004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26ea00640 1 -- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 msgr2=0x7fc2540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.216+0000 7fc26ea00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270071f20 0x7fc2540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.217+0000 7fc26ea00640 1 -- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254004800 msgr2=0x7fc254004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.217+0000 7fc26ea00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254004800 0x7fc254004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.217+0000 7fc26ea00640 1 -- 172.21.15.94:0/1760357149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc254003520 con 0x7fc254003a90 2024-05-27T12:36:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.217+0000 7fc267e00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254004800 0x7fc254004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:19.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.217+0000 7fc26ea00640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254003a90 0x7fc254003eb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c290 tx=0x7fc254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1daa1fcca240e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:19.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260007d50 con 0x7fc254003a90 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260005ad0 con 0x7fc254003a90 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc26e000640 1 -- 172.21.15.94:0/1760357149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260007070 con 0x7fc254003a90 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 -- 172.21.15.94:0/1760357149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254003a90 msgr2=0x7fc254003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 --2- 172.21.15.94:0/1760357149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254003a90 0x7fc254003eb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c290 tx=0x7fc254005830 comp rx=0 tx=0).stop 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc26f400640 1 -- 172.21.15.94:0/1760357149 reap_dead start 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 -- 172.21.15.94:0/1760357149 shutdown_connections 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 -- 172.21.15.94:0/1760357149 >> 172.21.15.94:0/1760357149 conn(0x7fc2700b3830 msgr2=0x7fc2700b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 -- 172.21.15.94:0/1760357149 shutdown_connections 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.218+0000 7fc275800640 1 -- 172.21.15.94:0/1760357149 wait complete. 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.220+0000 7fc275800640 1 Processor -- start 2024-05-27T12:36:19.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.220+0000 7fc275800640 1 -- start start 2024-05-27T12:36:19.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254003a90 0x7fc2700b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270139f20 0x7fc27013a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700ceb20 con 0x7fc27013e610 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700ce840 con 0x7fc254003a90 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.221+0000 7fc275800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc270145330 con 0x7fc270139f20 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46406/0 (socket says 172.21.15.94:46406) 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 -- 172.21.15.94:0/2132171644 learned_addr learned my addr 172.21.15.94:0/2132171644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc26ea00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254003a90 0x7fc2700b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc26f400640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270139f20 0x7fc27013a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 -- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270139f20 msgr2=0x7fc27013a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270139f20 0x7fc27013a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 -- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254003a90 msgr2=0x7fc2700b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254003a90 0x7fc2700b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.222+0000 7fc267e00640 1 -- 172.21.15.94:0/2132171644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27013ebd0 con 0x7fc27013e610 2024-05-27T12:36:19.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc267e00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc2580108e0 tx=0x7fc258010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f040 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25800ec20 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25801c400 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27013eeb0 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.223+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27013f470 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.224+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc270075460 con 0x7fc27013e610 2024-05-27T12:36:19.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.225+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc25801c5a0 con 0x7fc27013e610 2024-05-27T12:36:19.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.226+0000 7fc266a00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 0x7fc24c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:19.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.226+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc25808c3d0 con 0x7fc27013e610 2024-05-27T12:36:19.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.227+0000 7fc26ea00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 0x7fc24c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:19.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.228+0000 7fc26ea00640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 0x7fc24c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc260033000 tx=0x7fc260009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:19.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.231+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc258018030 con 0x7fc27013e610 2024-05-27T12:36:19.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.477+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc27013a8e0 con 0x7fc27013e610 2024-05-27T12:36:19.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.478+0000 7fc266a00640 1 -- 172.21.15.94:0/2132171644 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fc25805a760 con 0x7fc27013e610 2024-05-27T12:36:19.477 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T12:36:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 msgr2=0x7fc24c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 0x7fc24c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc260033000 tx=0x7fc260009030 comp rx=0 tx=0).stop 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 msgr2=0x7fc2700cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc2580108e0 tx=0x7fc258010da0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 shutdown_connections 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc24c06c8f0 0x7fc24c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc270139f20 0x7fc27013a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc27013e610 0x7fc2700cbfa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 --2- 172.21.15.94:0/2132171644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc254003a90 0x7fc2700b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:19.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.480+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 >> 172.21.15.94:0/2132171644 conn(0x7fc2700b3830 msgr2=0x7fc2700cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:19.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.481+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 shutdown_connections 2024-05-27T12:36:19.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19.481+0000 7fc275800640 1 -- 172.21.15.94:0/2132171644 wait complete. 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T12:36:19.497 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T12:36:19.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:19.498 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-05-27T12:36:19.548 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T12:36:19.548 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T12:36:19.548 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T12:36:19.548 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T12:36:19.549 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42490sec preferred_lft 42490sec 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether e6:74:08:05:41:39 brd ff:ff:ff:ff:ff:ff 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::f05f:10ff:fef1:11b0/64 scope link 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether e6:74:08:05:41:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::e474:8ff:fe05:4139/64 scope link 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.550 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T12:36:19.551 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether f6:17:ec:b3:f5:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-27T12:36:19.551 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::f417:ecff:feb3:f5c0/64 scope link 2024-05-27T12:36:19.551 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-05-27T12:36:19.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:19.552 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T12:36:19.627 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-05-27T12:36:19.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:19.631 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-05-27T12:36:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-05-27T12:36:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T12:36:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T12:36:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T12:36:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T12:36:19.679 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T12:36:19.680 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-05-27T12:36:19.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T12:36:19.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T12:36:19.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T12:36:19.680 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-05-27T12:36:19.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T12:36:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:36:19.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:19.883 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-05-27T12:36:19.930 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T12:36:19.930 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-27T12:36:19.931 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T12:36:19.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:19.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-05-27T12:36:20.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:36:20.011 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T12:36:20.062 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T12:36:20.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T12:36:20.063 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T12:36:20.117 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-27T12:36:20.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:20.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:36:20.236 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:36:20.236 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:36:20.271 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-27T12:36:20.271 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-27T12:36:20.272 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-05-27T12:36:20.272 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 0@b6a8d677-9e12-4245-aa4c-e7c6ae09d474.cephfs=/ 2024-05-27T12:36:20.272 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.94:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2024-05-27T12:36:20.274 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T12:36:20.274 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-05-27T12:36:20.274 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T12:36:20.274 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T12:36:20.274 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T12:36:20.279 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T12:36:20.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:20.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T12:36:20.347 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T12:36:20.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:36:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:36:20.474 INFO:teuthology.orchestra.run.smithi094.stdout:client4819 2024-05-27T12:36:20.475 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:20.475 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:20.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-27T12:36:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.785+0000 7f2cf6a00640 1 Processor -- start 2024-05-27T12:36:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.785+0000 7f2cf6a00640 1 -- start start 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cf6a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2cf0146060 0x7f2cf00b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 0x7f2cf0072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cf6a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf015cfd0 con 0x7f2cf00cd8a0 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cf6a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf015dbe0 con 0x7f2cf0146060 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf015d2b0 con 0x7f2cf006c040 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2cefe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f2cf0146060 0x7f2cf00b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2ce7400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f2cf00cd8a0 0x7f2cf006b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:55356/0 (socket says 172.21.15.94:55356) 2024-05-27T12:36:20.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.787+0000 7f2ce7400640 1 -- 172.21.15.94:0/1271658569 learned_addr learned my addr 172.21.15.94:0/1271658569 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cf4a00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 0x7f2cf0072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cefe00640 1 -- 172.21.15.94:0/1271658569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 msgr2=0x7f2cf0072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cefe00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 0x7f2cf0072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cefe00640 1 -- 172.21.15.94:0/1271658569 >> v1:172.21.15.94:6789/0 conn(0x7f2cf00cd8a0 legacy=0x7f2cf006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cef400640 1 -- 172.21.15.94:0/1271658569 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380164281 0 0) 0x7f2cf015cfd0 con 0x7f2cf00cd8a0 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cefe00640 1 -- 172.21.15.94:0/1271658569 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf015d590 con 0x7f2cf0146060 2024-05-27T12:36:20.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.788+0000 7f2cf4a00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 0x7f2cf0072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.789+0000 7f2cefe00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf0146060 0x7f2cf00b3f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00097a0 tx=0x7f2ce002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be39a8c4fe0a6842 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cef400640 1 -- 172.21.15.94:0/1271658569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0037070 con 0x7f2cf0146060 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cef400640 1 -- 172.21.15.94:0/1271658569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce0032440 con 0x7f2cf0146060 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cef400640 1 -- 172.21.15.94:0/1271658569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0032810 con 0x7f2cf0146060 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 -- 172.21.15.94:0/1271658569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf0146060 msgr2=0x7f2cf00b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:20.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf0146060 0x7f2cf00b3f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00097a0 tx=0x7f2ce002a180 comp rx=0 tx=0).stop 2024-05-27T12:36:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 -- 172.21.15.94:0/1271658569 shutdown_connections 2024-05-27T12:36:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf006c040 0x7f2cf0072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:20.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/1271658569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf0146060 0x7f2cf00b3f70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 -- 172.21.15.94:0/1271658569 >> 172.21.15.94:0/1271658569 conn(0x7f2cf00b2240 msgr2=0x7f2cf00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 -- 172.21.15.94:0/1271658569 shutdown_connections 2024-05-27T12:36:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.790+0000 7f2cf6a00640 1 -- 172.21.15.94:0/1271658569 wait complete. 2024-05-27T12:36:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.792+0000 7f2cf6a00640 1 Processor -- start 2024-05-27T12:36:20.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.792+0000 7f2cf6a00640 1 -- start start 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf00cd8a0 0x7f2cf01403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cf0146060 0x7f2cf015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf00d0110 con 0x7f2cf006c040 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0146440 con 0x7f2cf0146060 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cf6a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf00cdd30 con 0x7f2cf00cd8a0 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.793+0000 7f2cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32910/0 (socket says 172.21.15.94:32910) 2024-05-27T12:36:20.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 -- 172.21.15.94:0/2566815632 learned_addr learned my addr 172.21.15.94:0/2566815632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:20.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 -- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf00cd8a0 msgr2=0x7f2cf01403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:20.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf00cd8a0 0x7f2cf01403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 -- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cf0146060 msgr2=0x7f2cf015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cf0146060 0x7f2cf015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 -- 172.21.15.94:0/2566815632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf0140a00 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.794+0000 7f2cefe00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0035040 tx=0x7f2ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.795+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0037050 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.795+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce0038040 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.795+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf0140ce0 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.795+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004450 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.795+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01412a0 con 0x7f2cf006c040 2024-05-27T12:36:20.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.796+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf0068a70 con 0x7f2cf006c040 2024-05-27T12:36:20.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.797+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ce0030980 con 0x7f2cf006c040 2024-05-27T12:36:20.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.798+0000 7f2cecc00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 0x7f2ccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:20.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.798+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2ce00ac870 con 0x7f2cf006c040 2024-05-27T12:36:20.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.799+0000 7f2ce7400640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 0x7f2ccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:20.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.800+0000 7f2ce7400640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 0x7f2ccc06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8007fd0 tx=0x7f2cd8007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:20.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:20.803+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ce007ab80 con 0x7f2cf006c040 2024-05-27T12:36:21.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.052+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2cf006c660 con 0x7f2cf006c040 2024-05-27T12:36:21.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.483+0000 7f2cecc00640 1 -- 172.21.15.94:0/2566815632 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f2cf006c660 con 0x7f2cf006c040 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 msgr2=0x7f2ccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 0x7f2ccc06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8007fd0 tx=0x7f2cd8007920 comp rx=0 tx=0).stop 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 msgr2=0x7f2cf0141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0035040 tx=0x7f2ce00046c0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 shutdown_connections 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f2ccc06c920 0x7f2ccc06ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cf0146060 0x7f2cf015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cf00cd8a0 0x7f2cf01403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 --2- 172.21.15.94:0/2566815632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cf006c040 0x7f2cf0141aa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 >> 172.21.15.94:0/2566815632 conn(0x7f2cf00b2240 msgr2=0x7f2cf015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 shutdown_connections 2024-05-27T12:36:21.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.487+0000 7f2cf6a00640 1 -- 172.21.15.94:0/2566815632 wait complete. 2024-05-27T12:36:21.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-27T12:36:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.711+0000 7f5716e00640 1 Processor -- start 2024-05-27T12:36:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.711+0000 7f5716e00640 1 -- start start 2024-05-27T12:36:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f5716e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f57100b3f70 0x7f571006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f5716e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f571006be70 0x7f5710072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f5716e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571015cfd0 con 0x7f5710146060 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f5716e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f571015dbe0 con 0x7f57100b3f70 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f5716e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f571006c2c0 con 0x7f571006be70 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f570f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f57100b3f70 0x7f571006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f570f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f57100b3f70 0x7f571006b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32918/0 (socket says 172.21.15.94:32918) 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.713+0000 7f570f400640 1 -- 172.21.15.94:0/328053556 learned_addr learned my addr 172.21.15.94:0/328053556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.714+0000 7f570f400640 1 -- 172.21.15.94:0/328053556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f571006be70 msgr2=0x7f5710072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.714+0000 7f570f400640 1 --2- 172.21.15.94:0/328053556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f571006be70 0x7f5710072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.714+0000 7f570f400640 1 -- 172.21.15.94:0/328053556 >> v1:172.21.15.97:6789/0 conn(0x7f5710146060 legacy=0x7f57100cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:21.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.714+0000 7f570f400640 1 -- 172.21.15.94:0/328053556 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571015d2b0 con 0x7f57100b3f70 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.715+0000 7f570f400640 1 --2- 172.21.15.94:0/328053556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57100b3f70 0x7f571006b930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009870 tx=0x7f56fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434c36a2f3fe7fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.715+0000 7f570ea00640 1 -- 172.21.15.94:0/328053556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc037070 con 0x7f57100b3f70 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.715+0000 7f570ea00640 1 -- 172.21.15.94:0/328053556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc032440 con 0x7f57100b3f70 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.715+0000 7f570ea00640 1 -- 172.21.15.94:0/328053556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc032810 con 0x7f57100b3f70 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 -- 172.21.15.94:0/328053556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57100b3f70 msgr2=0x7f571006b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 --2- 172.21.15.94:0/328053556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57100b3f70 0x7f571006b930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009870 tx=0x7f56fc02a210 comp rx=0 tx=0).stop 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 -- 172.21.15.94:0/328053556 shutdown_connections 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 --2- 172.21.15.94:0/328053556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f571006be70 0x7f5710072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 --2- 172.21.15.94:0/328053556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57100b3f70 0x7f571006b930 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 -- 172.21.15.94:0/328053556 >> 172.21.15.94:0/328053556 conn(0x7f57100b2240 msgr2=0x7f57100b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 -- 172.21.15.94:0/328053556 shutdown_connections 2024-05-27T12:36:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.716+0000 7f5716e00640 1 -- 172.21.15.94:0/328053556 wait complete. 2024-05-27T12:36:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.718+0000 7f5716e00640 1 Processor -- start 2024-05-27T12:36:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.718+0000 7f5716e00640 1 -- start start 2024-05-27T12:36:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57100b3f70 0x7f57100ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5710146060 0x7f571015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57100cdfb0 con 0x7f571006be70 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f571015d870 con 0x7f57100b3f70 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5716e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5710146440 con 0x7f5710146060 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32926/0 (socket says 172.21.15.94:32926) 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 -- 172.21.15.94:0/60639403 learned_addr learned my addr 172.21.15.94:0/60639403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 -- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5710146060 msgr2=0x7f571015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f5714e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5710146060 0x7f571015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5710146060 0x7f571015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 -- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57100b3f70 msgr2=0x7f57100ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57100b3f70 0x7f57100ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.719+0000 7f570fe00640 1 -- 172.21.15.94:0/60639403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101f8980 con 0x7f571006be70 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f570fe00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f570000b3f0 tx=0x7f570000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700015020 con 0x7f571006be70 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5700011830 con 0x7f571006be70 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700011c20 con 0x7f571006be70 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101f8c60 con 0x7f571006be70 2024-05-27T12:36:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.720+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101f9220 con 0x7f571006be70 2024-05-27T12:36:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.721+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5710068a70 con 0x7f571006be70 2024-05-27T12:36:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.722+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5700018020 con 0x7f571006be70 2024-05-27T12:36:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.723+0000 7f5707e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 0x7f56f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.723+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f570008b800 con 0x7f571006be70 2024-05-27T12:36:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.724+0000 7f570f400640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 0x7f56f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.725+0000 7f570f400640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 0x7f56f006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56fc034000 tx=0x7f56fc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.728+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f570001f030 con 0x7f571006be70 2024-05-27T12:36:21.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.957+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f57100b3840 con 0x7f56f006c8f0 2024-05-27T12:36:21.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.961+0000 7f5707e00640 1 -- 172.21.15.94:0/60639403 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f57100b3840 con 0x7f56f006c8f0 2024-05-27T12:36:21.960 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:36:21.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:36:21.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-05-27T12:36:21.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:36:21.961 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:36:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 msgr2=0x7f56f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 0x7f56f006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56fc034000 tx=0x7f56fc02c000 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 msgr2=0x7f57100c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f570000b3f0 tx=0x7f570000b8b0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 shutdown_connections 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f56f006c8f0 0x7f56f006edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57100b3f70 0x7f57100ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5710146060 0x7f571015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 --2- 172.21.15.94:0/60639403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571006be70 0x7f57100c54b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 >> 172.21.15.94:0/60639403 conn(0x7f57100b2240 msgr2=0x7f571013ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 shutdown_connections 2024-05-27T12:36:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:21.964+0000 7f5716e00640 1 -- 172.21.15.94:0/60639403 wait complete. 2024-05-27T12:36:21.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-27T12:36:22.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.188+0000 7f9bafe00640 1 Processor -- start 2024-05-27T12:36:22.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.189+0000 7f9bafe00640 1 -- start start 2024-05-27T12:36:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb00ba3f0 0x7f9bb00b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9bafe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9bb00b66d0 0x7f9bb00d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9bafe00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb00b7840 con 0x7f9bb015e900 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9bafe00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb0067f30 con 0x7f9bb00b66d0 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9bafe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb0068210 con 0x7f9bb00ba3f0 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9baf400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f9bb00b66d0 0x7f9bb00d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9baea00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f9bb015e900 0x7f9bb0067630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:55362/0 (socket says 172.21.15.94:55362) 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.190+0000 7f9baea00640 1 -- 172.21.15.94:0/1674885526 learned_addr learned my addr 172.21.15.94:0/1674885526 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9bae000640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb00ba3f0 0x7f9bb00b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9bad600640 1 -- 172.21.15.94:0/1674885526 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266836571 0 0) 0x7f9ba0001110 con 0x7f9bb015e900 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9baf400640 1 -- 172.21.15.94:0/1674885526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb00ba3f0 msgr2=0x7f9bb00b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9baf400640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb00ba3f0 0x7f9bb00b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9baf400640 1 -- 172.21.15.94:0/1674885526 >> v1:172.21.15.94:6789/0 conn(0x7f9bb015e900 legacy=0x7f9bb0067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.191+0000 7f9baf400640 1 -- 172.21.15.94:0/1674885526 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00684f0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.192+0000 7f9baf400640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00d3310 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80097a0 tx=0x7f9ba802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d1a3b8ed635f98 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bad600640 1 -- 172.21.15.94:0/1674885526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8037070 con 0x7f9bb00b66d0 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bad600640 1 -- 172.21.15.94:0/1674885526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8032440 con 0x7f9bb00b66d0 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bad600640 1 -- 172.21.15.94:0/1674885526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8032810 con 0x7f9bb00b66d0 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 -- 172.21.15.94:0/1674885526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 msgr2=0x7f9bb00d3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00d3310 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80097a0 tx=0x7f9ba802a210 comp rx=0 tx=0).stop 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 -- 172.21.15.94:0/1674885526 shutdown_connections 2024-05-27T12:36:22.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00d3310 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 --2- 172.21.15.94:0/1674885526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb00ba3f0 0x7f9bb00b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.193+0000 7f9bafe00640 1 -- 172.21.15.94:0/1674885526 >> 172.21.15.94:0/1674885526 conn(0x7f9bb00d1190 msgr2=0x7f9bb00d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.194+0000 7f9bafe00640 1 -- 172.21.15.94:0/1674885526 shutdown_connections 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.194+0000 7f9bafe00640 1 -- 172.21.15.94:0/1674885526 wait complete. 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.195+0000 7f9bafe00640 1 Processor -- start 2024-05-27T12:36:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.195+0000 7f9bafe00640 1 -- start start 2024-05-27T12:36:22.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bb00ba3f0 0x7f9bb014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb015e900 0x7f9bb0155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb015ece0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00d2df0 con 0x7f9bb00ba3f0 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9bafe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00d3850 con 0x7f9bb015e900 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.196+0000 7f9baea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32942/0 (socket says 172.21.15.94:32942) 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 -- 172.21.15.94:0/2006937700 learned_addr learned my addr 172.21.15.94:0/2006937700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 -- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb015e900 msgr2=0x7f9bb0155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baf400640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb015e900 0x7f9bb0155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9bae000640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bb00ba3f0 0x7f9bb014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb015e900 0x7f9bb0155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 -- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bb00ba3f0 msgr2=0x7f9bb014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bb00ba3f0 0x7f9bb014f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 -- 172.21.15.94:0/2006937700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00cbd20 con 0x7f9bb00b66d0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.197+0000 7f9baea00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c430 tx=0x7f9ba000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.198+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00074e0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.198+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba00047e0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.198+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0004bd0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.198+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb014fe00 con 0x7f9bb00b66d0 2024-05-27T12:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.198+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb0156650 con 0x7f9bb00b66d0 2024-05-27T12:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.199+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0075460 con 0x7f9bb00b66d0 2024-05-27T12:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.200+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ba0011070 con 0x7f9bb00b66d0 2024-05-27T12:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.201+0000 7f9ba7400640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.201+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9ba008b8d0 con 0x7f9bb00b66d0 2024-05-27T12:36:22.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.202+0000 7f9bae000640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.203+0000 7f9bae000640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bb01613f0 tx=0x7f9b9800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.206+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bb00b7840 con 0x7f9bb00b66d0 2024-05-27T12:36:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.408+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9bb00d1570 con 0x7f9bb00b66d0 2024-05-27T12:36:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.409+0000 7f9ba7400640 1 -- 172.21.15.94:0/2006937700 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f9ba0059c60 con 0x7f9bb00b66d0 2024-05-27T12:36:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 msgr2=0x7f9b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9bb01613f0 tx=0x7f9b9800b040 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 msgr2=0x7f9bb00cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c430 tx=0x7f9ba000c8f0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 shutdown_connections 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bb00ba3f0 0x7f9bb014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9bb015e900 0x7f9bb0155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 --2- 172.21.15.94:0/2006937700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb00b66d0 0x7f9bb00cb7e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 >> 172.21.15.94:0/2006937700 conn(0x7f9bb00d1190 msgr2=0x7f9bb015c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 shutdown_connections 2024-05-27T12:36:22.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.412+0000 7f9bafe00640 1 -- 172.21.15.94:0/2006937700 wait complete. 2024-05-27T12:36:22.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-27T12:36:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.631+0000 7fbf4b200640 1 Processor -- start 2024-05-27T12:36:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.631+0000 7fbf4b200640 1 -- start start 2024-05-27T12:36:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.632+0000 7fbf4b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf440cdb30 0x7fbf440ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.632+0000 7fbf4b200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbf44072450 0x7fbf4415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.632+0000 7fbf4b200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf440cdf10 con 0x7fbf440ce6a0 2024-05-27T12:36:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.632+0000 7fbf4b200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440ceb40 con 0x7fbf44072450 2024-05-27T12:36:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.632+0000 7fbf4b200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fbf44160300 con 0x7fbf440cdb30 2024-05-27T12:36:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbf44072450 0x7fbf4415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf49400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fbf440ce6a0 0x7fbf44071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:55368/0 (socket says 172.21.15.94:55368) 2024-05-27T12:36:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fbf44072450 0x7fbf4415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32954/0 (socket says 172.21.15.94:32954) 2024-05-27T12:36:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf49400640 1 -- 172.21.15.94:0/347076141 learned_addr learned my addr 172.21.15.94:0/347076141 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf49e00640 1 --2- 172.21.15.94:0/347076141 >> v2:172.21.15.97:3300/0 conn(0x7fbf440cdb30 0x7fbf440ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 -- 172.21.15.94:0/347076141 >> v2:172.21.15.97:3300/0 conn(0x7fbf440cdb30 msgr2=0x7fbf440ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 --2- 172.21.15.94:0/347076141 >> v2:172.21.15.97:3300/0 conn(0x7fbf440cdb30 0x7fbf440ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 -- 172.21.15.94:0/347076141 >> v1:172.21.15.94:6789/0 conn(0x7fbf440ce6a0 legacy=0x7fbf44071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf4a800640 1 -- 172.21.15.94:0/347076141 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441605e0 con 0x7fbf44072450 2024-05-27T12:36:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.633+0000 7fbf48a00640 1 -- 172.21.15.94:0/347076141 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665109674 0 0) 0x7fbf440cdf10 con 0x7fbf440ce6a0 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.634+0000 7fbf4a800640 1 --2- 172.21.15.94:0/347076141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf4415d8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbf44068520 tx=0x7fbf4002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfc76f8e7de8c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf48a00640 1 -- 172.21.15.94:0/347076141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40007740 con 0x7fbf44072450 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf48a00640 1 -- 172.21.15.94:0/347076141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf400078e0 con 0x7fbf44072450 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf48a00640 1 -- 172.21.15.94:0/347076141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40007cb0 con 0x7fbf44072450 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf4b200640 1 -- 172.21.15.94:0/347076141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 msgr2=0x7fbf4415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf4b200640 1 --2- 172.21.15.94:0/347076141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf4415d8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbf44068520 tx=0x7fbf4002a680 comp rx=0 tx=0).stop 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.635+0000 7fbf4b200640 1 -- 172.21.15.94:0/347076141 shutdown_connections 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.636+0000 7fbf4b200640 1 --2- 172.21.15.94:0/347076141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf4415d8b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.636+0000 7fbf4b200640 1 --2- 172.21.15.94:0/347076141 >> v2:172.21.15.97:3300/0 conn(0x7fbf440cdb30 0x7fbf440ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.636+0000 7fbf4b200640 1 -- 172.21.15.94:0/347076141 >> 172.21.15.94:0/347076141 conn(0x7fbf44067e70 msgr2=0x7fbf4406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.636+0000 7fbf4b200640 1 -- 172.21.15.94:0/347076141 shutdown_connections 2024-05-27T12:36:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.636+0000 7fbf4b200640 1 -- 172.21.15.94:0/347076141 wait complete. 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.637+0000 7fbf4b200640 1 Processor -- start 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.637+0000 7fbf4b200640 1 -- start start 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbf440cdb30 0x7fbf441f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf440ce6a0 0x7fbf44145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44160f60 con 0x7fbf44072450 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44160de0 con 0x7fbf440ce6a0 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.638+0000 7fbf4b200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4415ddf0 con 0x7fbf440cdb30 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32966/0 (socket says 172.21.15.94:32966) 2024-05-27T12:36:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 -- 172.21.15.94:0/1220816910 learned_addr learned my addr 172.21.15.94:0/1220816910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 -- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbf440cdb30 msgr2=0x7fbf441f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbf440cdb30 0x7fbf441f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 -- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf440ce6a0 msgr2=0x7fbf44145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf440ce6a0 0x7fbf44145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 -- 172.21.15.94:0/1220816910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441f84f0 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.639+0000 7fbf49e00640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d830 tx=0x7fbf3800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.640+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800b6c0 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.640+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf38019070 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.640+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800ba80 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.640+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441492b0 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.640+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441f8a50 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.641+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4406efc0 con 0x7fbf44072450 2024-05-27T12:36:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.642+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf3801e460 con 0x7fbf44072450 2024-05-27T12:36:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.643+0000 7fbf3e000640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.643+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbf3808c240 con 0x7fbf44072450 2024-05-27T12:36:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.644+0000 7fbf49400640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.645+0000 7fbf49400640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 0x7fbf2806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007fd0 tx=0x7fbf34007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.647+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf4406efc0 con 0x7fbf44072450 2024-05-27T12:36:22.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.875+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fbf441567c0 con 0x7fbf2806c8f0 2024-05-27T12:36:22.886 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:36:22.887+0000 7fe60f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:36:22.886 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:36:22.887+0000 7fe60f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:36:22.886 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:36:22.887+0000 7fe60f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:36:22.887 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:36:22.887+0000 7fe60f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:36:22.887 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:36:22.887+0000 7fe60f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:36:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.971+0000 7fbf3e000640 1 -- 172.21.15.94:0/1220816910 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbf441567c0 con 0x7fbf2806c8f0 2024-05-27T12:36:22.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 msgr2=0x7fbf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 0x7fbf2806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007fd0 tx=0x7fbf34007960 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 msgr2=0x7fbf44150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d830 tx=0x7fbf3800dcf0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 shutdown_connections 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fbf2806c8f0 0x7fbf2806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf440ce6a0 0x7fbf44145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbf440cdb30 0x7fbf441f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 --2- 172.21.15.94:0/1220816910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf44072450 0x7fbf44150030 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 >> 172.21.15.94:0/1220816910 conn(0x7fbf44067e70 msgr2=0x7fbf4415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 shutdown_connections 2024-05-27T12:36:22.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:22.974+0000 7fbf4b200640 1 -- 172.21.15.94:0/1220816910 wait complete. 2024-05-27T12:36:22.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-27T12:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.199+0000 7fb3a3e00640 1 Processor -- start 2024-05-27T12:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.199+0000 7fb3a3e00640 1 -- start start 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.200+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a406ed60 0x7fb3a406f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.200+0000 7fb3a3e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3a40afba0 0x7fb3a4072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.200+0000 7fb3a3e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a40cf2f0 con 0x7fb3a4145020 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.200+0000 7fb3a3e00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a406f680 con 0x7fb3a40afba0 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.200+0000 7fb3a3e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a40cf590 con 0x7fb3a406ed60 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb39a000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fb3a4145020 0x7fb3a4071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:55372/0 (socket says 172.21.15.94:55372) 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a3400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fb3a40afba0 0x7fb3a4072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb39a000640 1 -- 172.21.15.94:0/3988409962 learned_addr learned my addr 172.21.15.94:0/3988409962 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a2a00640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a406ed60 0x7fb3a406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a3400640 1 -- 172.21.15.94:0/3988409962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a406ed60 msgr2=0x7fb3a406f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a3400640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a406ed60 0x7fb3a406f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a3400640 1 -- 172.21.15.94:0/3988409962 >> v1:172.21.15.94:6789/0 conn(0x7fb3a4145020 legacy=0x7fb3a4071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a3400640 1 -- 172.21.15.94:0/3988409962 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a40cf870 con 0x7fb3a40afba0 2024-05-27T12:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.201+0000 7fb3a2000640 1 -- 172.21.15.94:0/3988409962 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203923673 0 0) 0x7fb3a40cf2f0 con 0x7fb3a4145020 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.202+0000 7fb3a3400640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a40afba0 0x7fb3a4072430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0097a0 tx=0x7fb39c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedfe6469f926b0a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a2000640 1 -- 172.21.15.94:0/3988409962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c037070 con 0x7fb3a40afba0 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a2000640 1 -- 172.21.15.94:0/3988409962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c02ae50 con 0x7fb3a40afba0 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a2000640 1 -- 172.21.15.94:0/3988409962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c032660 con 0x7fb3a40afba0 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 -- 172.21.15.94:0/3988409962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a40afba0 msgr2=0x7fb3a4072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a40afba0 0x7fb3a4072430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0097a0 tx=0x7fb39c02a100 comp rx=0 tx=0).stop 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 -- 172.21.15.94:0/3988409962 shutdown_connections 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a40afba0 0x7fb3a4072430 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/3988409962 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a406ed60 0x7fb3a406f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 -- 172.21.15.94:0/3988409962 >> 172.21.15.94:0/3988409962 conn(0x7fb3a40b6230 msgr2=0x7fb3a40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 -- 172.21.15.94:0/3988409962 shutdown_connections 2024-05-27T12:36:23.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.203+0000 7fb3a3e00640 1 -- 172.21.15.94:0/3988409962 wait complete. 2024-05-27T12:36:23.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.205+0000 7fb3a3e00640 1 Processor -- start 2024-05-27T12:36:23.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.205+0000 7fb3a3e00640 1 -- start start 2024-05-27T12:36:23.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a40afba0 0x7fb3a4138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4145020 0x7fb3a4166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40b4c10 con 0x7fb3a406ed60 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40cfe30 con 0x7fb3a4145020 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a3e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a40b4d90 con 0x7fb3a40afba0 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32986/0 (socket says 172.21.15.94:32986) 2024-05-27T12:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.206+0000 7fb3a2a00640 1 -- 172.21.15.94:0/2396751240 learned_addr learned my addr 172.21.15.94:0/2396751240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:36:23.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb39a000640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a40afba0 0x7fb3a4138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 -- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a40afba0 msgr2=0x7fb3a4138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a40afba0 0x7fb3a4138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 -- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4145020 msgr2=0x7fb3a4166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a3400640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4145020 0x7fb3a4166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4145020 0x7fb3a4166dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 -- 172.21.15.94:0/2396751240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a41349e0 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a2a00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb394007e40 tx=0x7fb39400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394014620 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb394019070 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.207+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a4134420 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.208+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394014a00 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.208+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a41665c0 con 0x7fb3a406ed60 2024-05-27T12:36:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.209+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a4075460 con 0x7fb3a406ed60 2024-05-27T12:36:23.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.210+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb394014ba0 con 0x7fb3a406ed60 2024-05-27T12:36:23.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.211+0000 7fb39b400640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:36:23.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.211+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb39408c400 con 0x7fb3a406ed60 2024-05-27T12:36:23.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.211+0000 7fb39a000640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:36:23.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.212+0000 7fb39a000640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 0x7fb38006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007fd0 tx=0x7fb38c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:36:23.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.215+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb39405a820 con 0x7fb3a406ed60 2024-05-27T12:36:23.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.438+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb3a413a760 con 0x7fb38006c8f0 2024-05-27T12:36:23.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.453+0000 7fb39b400640 1 -- 172.21.15.94:0/2396751240 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb3a413a760 con 0x7fb38006c8f0 2024-05-27T12:36:23.452 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5 2024-05-27T12:36:23.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.455+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 msgr2=0x7fb38006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.455+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 0x7fb38006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007fd0 tx=0x7fb38c009040 comp rx=0 tx=0).stop 2024-05-27T12:36:23.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.455+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 msgr2=0x7fb3a415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.455+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb394007e40 tx=0x7fb39400e4b0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 shutdown_connections 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fb38006c8f0 0x7fb38006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3a4145020 0x7fb3a4166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb3a40afba0 0x7fb3a4138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 --2- 172.21.15.94:0/2396751240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a406ed60 0x7fb3a415dfc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 >> 172.21.15.94:0/2396751240 conn(0x7fb3a40b6230 msgr2=0x7fb3a4139800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 shutdown_connections 2024-05-27T12:36:23.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:36:23.456+0000 7fb3a3e00640 1 -- 172.21.15.94:0/2396751240 wait complete. 2024-05-27T12:36:23.472 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5 2024-05-27T12:36:23.472 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:23.473 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:23.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:23.519 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:36:28.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:36:28.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:36:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:36:30.864 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:36:30.864 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.23998 s, 148 MB/s 2024-05-27T12:36:30.865 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:30.865 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:30.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:30.905 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:36:38.274 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:36:38.274 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:36:38.274 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.28076 s, 147 MB/s 2024-05-27T12:36:38.275 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:38.276 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:38.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:38.317 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:36:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:36:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:36:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.2397 s, 148 MB/s 2024-05-27T12:36:45.648 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:45.648 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:45.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:45.685 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:36:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:36:53.228 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:36:53.228 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.45604 s, 144 MB/s 2024-05-27T12:36:53.229 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:36:53.229 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:36:53.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:36:53.268 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:36:58.944 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:36:58.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:37:00.651 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.29724 s, 147 MB/s 2024-05-27T12:37:00.653 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:37:00.653 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:37:00.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:00.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:37:08.179 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:08.179 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:08.179 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.40274 s, 145 MB/s 2024-05-27T12:37:08.180 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:37:08.180 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:37:08.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:08.219 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:37:15.632 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:15.632 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:15.633 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.32614 s, 147 MB/s 2024-05-27T12:37:15.634 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:37:15.634 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:37:15.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:15.679 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:37:23.211 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:23.211 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:23.211 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.44229 s, 144 MB/s 2024-05-27T12:37:23.213 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:37:23.213 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:37:23.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:23.255 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:37:29.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:37:29.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:37:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:30.626 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.27736 s, 148 MB/s 2024-05-27T12:37:30.627 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:37:30.627 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:37:30.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:30.665 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-27T12:37:38.076 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records in 2024-05-27T12:37:38.076 INFO:teuthology.orchestra.run.smithi094.stderr:1024+0 records out 2024-05-27T12:37:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.32052 s, 147 MB/s 2024-05-27T12:37:38.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-27T12:37:38.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.252+0000 7fa395200640 1 Processor -- start 2024-05-27T12:37:38.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.252+0000 7fa395200640 1 -- start start 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa395200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa39015cf50 0x7fa3901642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa395200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa390071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa395200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39006fb60 con 0x7fa390072280 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa395200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa3900cc980 con 0x7fa39015cf50 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa395200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa39006fe40 con 0x7fa39015d330 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa38fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa390072280 0x7fa39006ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44006/0 (socket says 172.21.15.94:44006) 2024-05-27T12:37:38.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.254+0000 7fa38fe00640 1 -- 172.21.15.94:0/387570473 learned_addr learned my addr 172.21.15.94:0/387570473 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> v2:172.21.15.97:3300/0 conn(0x7fa39015cf50 0x7fa3901642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa387e00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa390071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020270226 0 0) 0x7fa39006fb60 con 0x7fa390072280 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa370003520 con 0x7fa390072280 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6289615 0 0) 0x7fa370003520 con 0x7fa390072280 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 msgr2=0x7fa390071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa390071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 >> v2:172.21.15.97:3300/0 conn(0x7fa39015cf50 msgr2=0x7fa3901642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:38.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 --2- 172.21.15.94:0/387570473 >> v2:172.21.15.97:3300/0 conn(0x7fa39015cf50 0x7fa3901642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3900ce460 con 0x7fa390072280 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fa3880019f0 con 0x7fa390072280 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 >> v1:172.21.15.94:6789/0 conn(0x7fa390072280 legacy=0x7fa39006ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3700039b0 0x7fa370003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa370004800 0x7fa370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa3700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa39006fe40 con 0x7fa3700039b0 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa37000b9f0 con 0x7fa370004800 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa37000bbf0 con 0x7fa39015d330 2024-05-27T12:37:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.255+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> v2:172.21.15.97:3300/0 conn(0x7fa39015cf50 0x7fa3901642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3700039b0 0x7fa370003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa387e00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa370004800 0x7fa370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38fe00640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa3700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 -- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 msgr2=0x7fa3700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa39015d330 0x7fa3700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 -- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa370004800 msgr2=0x7fa370004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa370004800 0x7fa370004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.256+0000 7fa38f400640 1 -- 172.21.15.94:0/387570473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa370003520 con 0x7fa3700039b0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.257+0000 7fa38f400640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3700039b0 0x7fa370003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd79dc78a10f7c1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.257+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380037070 con 0x7fa3700039b0 2024-05-27T12:37:38.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.257+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38002ace0 con 0x7fa3700039b0 2024-05-27T12:37:38.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.257+0000 7fa38ea00640 1 -- 172.21.15.94:0/387570473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38000f040 con 0x7fa3700039b0 2024-05-27T12:37:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 -- 172.21.15.94:0/387570473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3700039b0 msgr2=0x7fa370003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 --2- 172.21.15.94:0/387570473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3700039b0 0x7fa370003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa370005830 comp rx=0 tx=0).stop 2024-05-27T12:37:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa38fe00640 1 -- 172.21.15.94:0/387570473 reap_dead start 2024-05-27T12:37:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 -- 172.21.15.94:0/387570473 shutdown_connections 2024-05-27T12:37:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 -- 172.21.15.94:0/387570473 >> 172.21.15.94:0/387570473 conn(0x7fa39006b930 msgr2=0x7fa39006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 -- 172.21.15.94:0/387570473 shutdown_connections 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.258+0000 7fa395200640 1 -- 172.21.15.94:0/387570473 wait complete. 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.260+0000 7fa395200640 1 Processor -- start 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.260+0000 7fa395200640 1 -- start start 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3700039b0 0x7fa39015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3901560d0 0x7fa39014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900ce220 con 0x7fa39015f9d0 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900ce0a0 con 0x7fa3901560d0 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa395200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa390164820 con 0x7fa3700039b0 2024-05-27T12:37:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52574/0 (socket says 172.21.15.94:52574) 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 -- 172.21.15.94:0/1172465773 learned_addr learned my addr 172.21.15.94:0/1172465773 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 -- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3700039b0 msgr2=0x7fa39015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3700039b0 0x7fa39015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa387e00640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3901560d0 0x7fa39014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 -- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3901560d0 msgr2=0x7fa39014e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3901560d0 0x7fa39014e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.261+0000 7fa38fe00640 1 -- 172.21.15.94:0/1172465773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39014eb20 con 0x7fa39015f9d0 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa38fe00640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa3900cffc0 tx=0x7fa38800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38800ebc0 con 0x7fa39015f9d0 2024-05-27T12:37:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38800ed80 con 0x7fa39015f9d0 2024-05-27T12:37:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388014670 con 0x7fa39015f9d0 2024-05-27T12:37:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa390154ec0 con 0x7fa39015f9d0 2024-05-27T12:37:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.262+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa390155480 con 0x7fa39015f9d0 2024-05-27T12:37:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.263+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3901564b0 con 0x7fa39015f9d0 2024-05-27T12:37:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.264+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa388014810 con 0x7fa39015f9d0 2024-05-27T12:37:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.265+0000 7fa387400640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 0x7fa36c06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.265+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa38808ba30 con 0x7fa39015f9d0 2024-05-27T12:37:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.266+0000 7fa38f400640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 0x7fa36c06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.267+0000 7fa38f400640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 0x7fa36c06ee90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa380034000 tx=0x7fa38002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.270+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa388059e80 con 0x7fa39015f9d0 2024-05-27T12:37:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.486+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fa390160390 con 0x7fa36c06c9d0 2024-05-27T12:37:38.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.523+0000 7fa387400640 1 -- 172.21.15.94:0/1172465773 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa390160390 con 0x7fa36c06c9d0 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 msgr2=0x7fa36c06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 0x7fa36c06ee90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa380034000 tx=0x7fa38002c000 comp rx=0 tx=0).stop 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 msgr2=0x7fa39015fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa3900cffc0 tx=0x7fa38800e530 comp rx=0 tx=0).stop 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 shutdown_connections 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa36c06c9d0 0x7fa36c06ee90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3901560d0 0x7fa39014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa39015f9d0 0x7fa39015fe50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 --2- 172.21.15.94:0/1172465773 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3700039b0 0x7fa39015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 >> 172.21.15.94:0/1172465773 conn(0x7fa39006b930 msgr2=0x7fa3900b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:38.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 shutdown_connections 2024-05-27T12:37:38.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.526+0000 7fa395200640 1 -- 172.21.15.94:0/1172465773 wait complete. 2024-05-27T12:37:38.544 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-27T12:37:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.722+0000 7fd9b6a00640 1 Processor -- start 2024-05-27T12:37:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.722+0000 7fd9b6a00640 1 -- start start 2024-05-27T12:37:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b013f480 0x7fd9b0071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9b0072260 0x7fd9b015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b006dc50 con 0x7fd9b015d120 2024-05-27T12:37:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b00726c0 con 0x7fd9b0072260 2024-05-27T12:37:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b013fcd0 con 0x7fd9b013f480 2024-05-27T12:37:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9b0072260 0x7fd9b015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fd9b0072260 0x7fd9b015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52590/0 (socket says 172.21.15.94:52590) 2024-05-27T12:37:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b6000640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd9b015d120 0x7fd9b0141d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:44020/0 (socket says 172.21.15.94:44020) 2024-05-27T12:37:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 -- 172.21.15.94:0/3303701252 learned_addr learned my addr 172.21.15.94:0/3303701252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 -- 172.21.15.94:0/3303701252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b013f480 msgr2=0x7fd9b0071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b5600640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b013f480 0x7fd9b0071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b013f480 0x7fd9b0071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 -- 172.21.15.94:0/3303701252 >> v1:172.21.15.94:6789/0 conn(0x7fd9b015d120 legacy=0x7fd9b0141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9b4c00640 1 -- 172.21.15.94:0/3303701252 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b015d950 con 0x7fd9b0072260 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.724+0000 7fd9afe00640 1 -- 172.21.15.94:0/3303701252 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024759343 0 0) 0x7fd9a8001490 con 0x7fd9b015d120 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.725+0000 7fd9b4c00640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b015cbe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009870 tx=0x7fd99c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f7913868e33f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9afe00640 1 -- 172.21.15.94:0/3303701252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c037070 con 0x7fd9b0072260 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9afe00640 1 -- 172.21.15.94:0/3303701252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c032440 con 0x7fd9b0072260 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9afe00640 1 -- 172.21.15.94:0/3303701252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c032810 con 0x7fd9b0072260 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 -- 172.21.15.94:0/3303701252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 msgr2=0x7fd9b015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b015cbe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009870 tx=0x7fd99c02a270 comp rx=0 tx=0).stop 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 -- 172.21.15.94:0/3303701252 shutdown_connections 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b015cbe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/3303701252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b013f480 0x7fd9b0071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 -- 172.21.15.94:0/3303701252 >> 172.21.15.94:0/3303701252 conn(0x7fd9b00b6230 msgr2=0x7fd9b00b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 -- 172.21.15.94:0/3303701252 shutdown_connections 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.726+0000 7fd9b6a00640 1 -- 172.21.15.94:0/3303701252 wait complete. 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.728+0000 7fd9b6a00640 1 Processor -- start 2024-05-27T12:37:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.728+0000 7fd9b6a00640 1 -- start start 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b013f480 0x7fd9b00b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b015d120 0x7fd9b0140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0140200 con 0x7fd9b0072260 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0140570 con 0x7fd9b013f480 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b6a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b0141a10 con 0x7fd9b015d120 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52606/0 (socket says 172.21.15.94:52606) 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b5600640 1 -- 172.21.15.94:0/1474846017 learned_addr learned my addr 172.21.15.94:0/1474846017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b5600640 1 -- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b015d120 msgr2=0x7fd9b0140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.729+0000 7fd9b5600640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b015d120 0x7fd9b0140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b5600640 1 -- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b013f480 msgr2=0x7fd9b00b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b5600640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b013f480 0x7fd9b00b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b5600640 1 -- 172.21.15.94:0/1474846017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b0156240 con 0x7fd9b0072260 2024-05-27T12:37:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b5600640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c820 tx=0x7fd9a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4015070 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a4004020 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40043f0 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b00b3d80 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.730+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b00afdd0 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.731+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b0075460 con 0x7fd9b0072260 2024-05-27T12:37:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.732+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd9a4007830 con 0x7fd9b0072260 2024-05-27T12:37:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.733+0000 7fd9ad600640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 0x7fd99006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.733+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd9a408bbd0 con 0x7fd9b0072260 2024-05-27T12:37:38.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.734+0000 7fd9b4c00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 0x7fd99006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.735+0000 7fd9b4c00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 0x7fd99006ee70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd99c034000 tx=0x7fd99c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.737+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9a4059e90 con 0x7fd9b0072260 2024-05-27T12:37:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.957+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd9b0072f30 con 0x7fd99006c9b0 2024-05-27T12:37:38.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.969+0000 7fd9ad600640 1 -- 172.21.15.94:0/1474846017 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd9b0072f30 con 0x7fd99006c9b0 2024-05-27T12:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5 2024-05-27T12:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.971+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 msgr2=0x7fd99006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.971+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 0x7fd99006ee70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd99c034000 tx=0x7fd99c02c000 comp rx=0 tx=0).stop 2024-05-27T12:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 msgr2=0x7fd9b0155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c820 tx=0x7fd9a400cce0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 shutdown_connections 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd99006c9b0 0x7fd99006ee70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9b013f480 0x7fd9b00b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9b015d120 0x7fd9b0140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 --2- 172.21.15.94:0/1474846017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9b0072260 0x7fd9b0155d00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 >> 172.21.15.94:0/1474846017 conn(0x7fd9b00b6230 msgr2=0x7fd9b006e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 shutdown_connections 2024-05-27T12:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:38.972+0000 7fd9b6a00640 1 -- 172.21.15.94:0/1474846017 wait complete. 2024-05-27T12:37:38.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:37:38.989 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5 2024-05-27T12:37:38.989 DEBUG:teuthology.orchestra.run.smithi094:> ) 2024-05-27T12:37:39.032 INFO:teuthology.orchestra.run.smithi094.stdout:# file: volumes/_nogroup/sv1/9fe8b901-dc23-4ae9-aad4-9f2f52baf2a5 2024-05-27T12:37:39.032 INFO:teuthology.orchestra.run.smithi094.stdout:ceph.dir.rbytes="10737418240" 2024-05-27T12:37:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:39.034 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-27T12:37:39.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-27T12:37:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.282+0000 7f8d11c00640 1 Processor -- start 2024-05-27T12:37:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.283+0000 7f8d11c00640 1 -- start start 2024-05-27T12:37:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.284+0000 7f8d11c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c1624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.284+0000 7f8d11c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d0c0adb10 0x7f8d0c162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.284+0000 7f8d11c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0c15ed10 con 0x7f8d0c15e930 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.284+0000 7f8d11c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c072930 con 0x7f8d0c0adb10 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.284+0000 7f8d11c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c072000 con 0x7f8d0c0b4710 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d0c0adb10 0x7f8d0c162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f8d0c0adb10 0x7f8d0c162a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52612/0 (socket says 172.21.15.94:52612) 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 -- 172.21.15.94:0/276759309 learned_addr learned my addr 172.21.15.94:0/276759309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d03e00640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c1624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 -- 172.21.15.94:0/276759309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 msgr2=0x7f8d0c1624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c1624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 -- 172.21.15.94:0/276759309 >> v1:172.21.15.97:6789/0 conn(0x7f8d0c15e930 legacy=0x7f8d0c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d0b400640 1 -- 172.21.15.94:0/276759309 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c0722e0 con 0x7f8d0c0adb10 2024-05-27T12:37:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.285+0000 7f8d03e00640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c1624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.286+0000 7f8d0b400640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c162a00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5273965efe2e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d0a000640 1 -- 172.21.15.94:0/276759309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04036070 con 0x7f8d0c0adb10 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d0a000640 1 -- 172.21.15.94:0/276759309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04031600 con 0x7f8d0c0adb10 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d0a000640 1 -- 172.21.15.94:0/276759309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040319d0 con 0x7f8d0c0adb10 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 -- 172.21.15.94:0/276759309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c0adb10 msgr2=0x7f8d0c162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c162a00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0402a710 comp rx=0 tx=0).stop 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 -- 172.21.15.94:0/276759309 shutdown_connections 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c162a00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 --2- 172.21.15.94:0/276759309 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c1624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 -- 172.21.15.94:0/276759309 >> 172.21.15.94:0/276759309 conn(0x7f8d0c0b2240 msgr2=0x7f8d0c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 -- 172.21.15.94:0/276759309 shutdown_connections 2024-05-27T12:37:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.287+0000 7f8d11c00640 1 -- 172.21.15.94:0/276759309 wait complete. 2024-05-27T12:37:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.289+0000 7f8d11c00640 1 Processor -- start 2024-05-27T12:37:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.289+0000 7f8d11c00640 1 -- start start 2024-05-27T12:37:39.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b3850 con 0x7f8d0c15e930 2024-05-27T12:37:39.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0b36d0 con 0x7f8d0c0adb10 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.290+0000 7f8d11c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c06fd20 con 0x7f8d0c0b4710 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52614/0 (socket says 172.21.15.94:52614) 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 -- 172.21.15.94:0/1198417898 learned_addr learned my addr 172.21.15.94:0/1198417898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d03e00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 -- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 msgr2=0x7f8d0c147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 -- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 msgr2=0x7f8d0c1479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0aa00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c1479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 -- 172.21.15.94:0/1198417898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1e7c10 con 0x7f8d0c15e930 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0aa00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.291+0000 7f8d0b400640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009f70 tx=0x7f8d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.292+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0403f020 con 0x7f8d0c15e930 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.292+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04037040 con 0x7f8d0c15e930 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.292+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04036070 con 0x7f8d0c15e930 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.292+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c15be70 con 0x7f8d0c15e930 2024-05-27T12:37:39.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.292+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c154250 con 0x7f8d0c15e930 2024-05-27T12:37:39.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.293+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c068a70 con 0x7f8d0c15e930 2024-05-27T12:37:39.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.294+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8d04042020 con 0x7f8d0c15e930 2024-05-27T12:37:39.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.295+0000 7f8d02a00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 0x7f8ce806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.295+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8d040ac2b0 con 0x7f8d0c15e930 2024-05-27T12:37:39.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.296+0000 7f8d0aa00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 0x7f8ce806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.297+0000 7f8d0aa00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 0x7f8ce806ee40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0015e0 tx=0x7f8cfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.300+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d04049030 con 0x7f8d0c15e930 2024-05-27T12:37:39.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.534+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f8d0c15e150 con 0x7f8ce806c980 2024-05-27T12:37:39.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.713+0000 7f8d02a00640 1 -- 172.21.15.94:0/1198417898 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d0c15e150 con 0x7f8ce806c980 2024-05-27T12:37:39.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.716+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 msgr2=0x7f8ce806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 0x7f8ce806ee40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0015e0 tx=0x7f8cfc009040 comp rx=0 tx=0).stop 2024-05-27T12:37:39.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 msgr2=0x7f8d0c1f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009f70 tx=0x7f8d04004a40 comp rx=0 tx=0).stop 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 shutdown_connections 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8ce806c980 0x7f8ce806ee40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d0c15e930 0x7f8d0c1f8540 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d0c0b4710 0x7f8d0c147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 --2- 172.21.15.94:0/1198417898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c0adb10 0x7f8d0c1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 >> 172.21.15.94:0/1198417898 conn(0x7f8d0c0b2240 msgr2=0x7f8d0c0b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 shutdown_connections 2024-05-27T12:37:39.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.717+0000 7f8d11c00640 1 -- 172.21.15.94:0/1198417898 wait complete. 2024-05-27T12:37:39.719 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.720+0000 7fe615e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.719 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.720+0000 7fe615e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.719 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.720+0000 7fe615e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.719 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.720+0000 7fe615e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.719 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.720+0000 7fe615e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:39.748 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.749+0000 7fe614a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.748 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.749+0000 7fe614a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.748 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.749+0000 7fe614a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.749 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.749+0000 7fe614a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.749 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:37:39.749+0000 7fe614a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.932+0000 7fd1c3000640 1 Processor -- start 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.932+0000 7fd1c3000640 1 -- start start 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.933+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 0x7fd1bc071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c3000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc13e9e0 con 0x7fd1bc0cd480 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c3000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1bc15ba50 con 0x7fd1bc13e600 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c3000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc15bd10 con 0x7fd1bc06f5e0 2024-05-27T12:37:39.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c2600640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fd1bc0cd480 0x7fd1bc0cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51694/0 (socket says 172.21.15.94:51694) 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c2600640 1 -- 172.21.15.94:0/3608746048 learned_addr learned my addr 172.21.15.94:0/3608746048 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1c1200640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 0x7fd1bc071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489710796 0 0) 0x7fd1b0001110 con 0x7fd1bc0cd480 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.934+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1a4005730 con 0x7fd1bc0cd480 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1671960235 0 0) 0x7fd1a4005730 con 0x7fd1bc0cd480 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 msgr2=0x7fd1bc071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 0x7fd1bc071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 >> v1:172.21.15.97:6789/0 conn(0x7fd1bc13e600 legacy=0x7fd1bc06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc15bff0 con 0x7fd1bc0cd480 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fd1b0001840 con 0x7fd1bc0cd480 2024-05-27T12:37:39.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 >> v1:172.21.15.94:6789/0 conn(0x7fd1bc0cd480 legacy=0x7fd1bc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4005bf0 0x7fd1a4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 0x7fd1a4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a4006f50 0x7fd1a40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc15bd10 con 0x7fd1a4006f50 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a400dd60 con 0x7fd1a4005bf0 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a400df60 con 0x7fd1bc06f5e0 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.935+0000 7fd1c2600640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a4006f50 0x7fd1a40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c1c00640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4005bf0 0x7fd1a4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c2600640 1 -- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 msgr2=0x7fd1a4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c2600640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc06f5e0 0x7fd1a4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c2600640 1 -- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4005bf0 msgr2=0x7fd1a4006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:39.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c2600640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4005bf0 0x7fd1a4006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.936+0000 7fd1c2600640 1 -- 172.21.15.94:0/3608746048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a4005730 con 0x7fd1a4006f50 2024-05-27T12:37:39.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1c2600640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a4006f50 0x7fd1a40073d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0007fd0 tx=0x7fd1a4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a4db84600e9a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0007e30 con 0x7fd1a4006f50 2024-05-27T12:37:39.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0005af0 con 0x7fd1a4006f50 2024-05-27T12:37:39.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1bbe00640 1 -- 172.21.15.94:0/3608746048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0007070 con 0x7fd1a4006f50 2024-05-27T12:37:39.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1c3000640 1 -- 172.21.15.94:0/3608746048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a4006f50 msgr2=0x7fd1a40073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1c3000640 1 --2- 172.21.15.94:0/3608746048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1a4006f50 0x7fd1a40073d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0007fd0 tx=0x7fd1a4007a40 comp rx=0 tx=0).stop 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.937+0000 7fd1c2600640 1 -- 172.21.15.94:0/3608746048 reap_dead start 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.938+0000 7fd1c3000640 1 -- 172.21.15.94:0/3608746048 shutdown_connections 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.938+0000 7fd1c3000640 1 -- 172.21.15.94:0/3608746048 >> 172.21.15.94:0/3608746048 conn(0x7fd1bc0b2160 msgr2=0x7fd1bc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.938+0000 7fd1c3000640 1 -- 172.21.15.94:0/3608746048 shutdown_connections 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.938+0000 7fd1c3000640 1 -- 172.21.15.94:0/3608746048 wait complete. 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.939+0000 7fd1c3000640 1 Processor -- start 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 -- start start 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 0x7fd1bc0c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc150740 0x7fd1bc1f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc15c920 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc15c5b0 con 0x7fd1a4006f50 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.940+0000 7fd1c3000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc071d60 con 0x7fd1bc150740 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56838/0 (socket says 172.21.15.94:56838) 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 -- 172.21.15.94:0/2127657272 learned_addr learned my addr 172.21.15.94:0/2127657272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c1c00640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 0x7fd1bc0c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c1200640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc150740 0x7fd1bc1f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 -- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc150740 msgr2=0x7fd1bc1f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc150740 0x7fd1bc1f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 -- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 msgr2=0x7fd1bc0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 0x7fd1bc0c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 -- 172.21.15.94:0/2127657272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc0c66e0 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c1c00640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 0x7fd1bc0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.941+0000 7fd1c2600640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c3e0 tx=0x7fd1b0001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.942+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0000ed0 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.942+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0042070 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.942+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b003b620 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.942+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc0c69c0 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.942+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc14df30 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.943+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc14e130 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.944+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd1b0038040 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.945+0000 7fd1b9600640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 0x7fd19c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:39.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.945+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd1b00ac5b0 con 0x7fd1bc1f47f0 2024-05-27T12:37:39.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.945+0000 7fd1c1c00640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 0x7fd19c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:39.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.946+0000 7fd1c1c00640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 0x7fd19c06f010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fd0 tx=0x7fd1b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:39.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:39.950+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1bc14e130 con 0x7fd1bc1f47f0 2024-05-27T12:37:40.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.222+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd1bc075460 con 0x7fd1bc1f47f0 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.223+0000 7fd1b9600640 1 -- 172.21.15.94:0/2127657272 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1830 (secure 0 0 0) 0x7fd1bc13e9e0 con 0x7fd1bc1f47f0 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:40.223 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:40.224 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 180, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:40.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2447, 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 10159721714, 2024-05-27T12:37:40.226 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 20204732416, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 752889380864, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 162829381, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 40 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:40.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:40.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:40.229 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:40.230 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:40.230 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:40.230 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:40.230 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:40.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": {} 2024-05-27T12:37:40.231 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 msgr2=0x7fd19c06f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 0x7fd19c06f010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fd0 tx=0x7fd1b4009040 comp rx=0 tx=0).stop 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 msgr2=0x7fd1bc1401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c3e0 tx=0x7fd1b0001b40 comp rx=0 tx=0).stop 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 shutdown_connections 2024-05-27T12:37:40.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fd19c06cb50 0x7fd19c06f010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1bc1f47f0 0x7fd1bc1401d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1bc150740 0x7fd1bc1f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 --2- 172.21.15.94:0/2127657272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1a4006f50 0x7fd1bc0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.226+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 >> 172.21.15.94:0/2127657272 conn(0x7fd1bc0b2160 msgr2=0x7fd1bc15d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.227+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 shutdown_connections 2024-05-27T12:37:40.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:40.227+0000 7fd1c3000640 1 -- 172.21.15.94:0/2127657272 wait complete. 2024-05-27T12:37:42.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.450+0000 7ff75ca00640 1 Processor -- start 2024-05-27T12:37:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.450+0000 7ff75ca00640 1 -- start start 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff75ca00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff75815ea30 0x7ff7580d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff75ca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7580b4510 0x7ff7580d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff75ca00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75815ee10 con 0x7ff7580d4910 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff75ca00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7ff7580d2830 con 0x7ff75815ea30 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff75ca00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff7580d2a60 con 0x7ff7580b4510 2024-05-27T12:37:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff756800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff75815ea30 0x7ff7580d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff756800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7ff75815ea30 0x7ff7580d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56840/0 (socket says 172.21.15.94:56840) 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff756800640 1 -- 172.21.15.94:0/2553709611 learned_addr learned my addr 172.21.15.94:0/2553709611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff756800640 1 -- 172.21.15.94:0/2553709611 >> v2:172.21.15.97:3300/0 conn(0x7ff7580b4510 msgr2=0x7ff7580d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff74de00640 1 --2- 172.21.15.94:0/2553709611 >> v2:172.21.15.97:3300/0 conn(0x7ff7580b4510 0x7ff7580d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.452+0000 7ff756800640 1 --2- 172.21.15.94:0/2553709611 >> v2:172.21.15.97:3300/0 conn(0x7ff7580b4510 0x7ff7580d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.453+0000 7ff756800640 1 -- 172.21.15.94:0/2553709611 >> v1:172.21.15.97:6789/0 conn(0x7ff7580d4910 legacy=0x7ff7580d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.453+0000 7ff756800640 1 -- 172.21.15.94:0/2553709611 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580d2d40 con 0x7ff75815ea30 2024-05-27T12:37:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.453+0000 7ff756800640 1 --2- 172.21.15.94:0/2553709611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580d22f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff75808cd60 tx=0x7ff74802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62384127ad9bb8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff755e00640 1 -- 172.21.15.94:0/2553709611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748037070 con 0x7ff75815ea30 2024-05-27T12:37:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff755e00640 1 -- 172.21.15.94:0/2553709611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff748032440 con 0x7ff75815ea30 2024-05-27T12:37:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff755e00640 1 -- 172.21.15.94:0/2553709611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748032810 con 0x7ff75815ea30 2024-05-27T12:37:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff75ca00640 1 -- 172.21.15.94:0/2553709611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 msgr2=0x7ff7580d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff75ca00640 1 --2- 172.21.15.94:0/2553709611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580d22f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff75808cd60 tx=0x7ff74802a2c0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff75ca00640 1 -- 172.21.15.94:0/2553709611 shutdown_connections 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff75ca00640 1 --2- 172.21.15.94:0/2553709611 >> v2:172.21.15.97:3300/0 conn(0x7ff7580b4510 0x7ff7580d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.454+0000 7ff75ca00640 1 --2- 172.21.15.94:0/2553709611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580d22f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.455+0000 7ff75ca00640 1 -- 172.21.15.94:0/2553709611 >> 172.21.15.94:0/2553709611 conn(0x7ff75809da70 msgr2=0x7ff75808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.455+0000 7ff75ca00640 1 -- 172.21.15.94:0/2553709611 shutdown_connections 2024-05-27T12:37:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.455+0000 7ff75ca00640 1 -- 172.21.15.94:0/2553709611 wait complete. 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.456+0000 7ff75ca00640 1 Processor -- start 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.456+0000 7ff75ca00640 1 -- start start 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7580b4510 0x7ff7581f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7580d4910 0x7ff7580ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7580d35e0 con 0x7ff75815ea30 2024-05-27T12:37:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff7580d3300 con 0x7ff7580d4910 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.457+0000 7ff75ca00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7ff7580d4d20 con 0x7ff7580b4510 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56842/0 (socket says 172.21.15.94:56842) 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 -- 172.21.15.94:0/1159299369 learned_addr learned my addr 172.21.15.94:0/1159299369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff74de00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7580d4910 0x7ff7580ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 -- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7580b4510 msgr2=0x7ff7581f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff756800640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7580b4510 0x7ff7581f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7580b4510 0x7ff7581f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 -- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7580d4910 msgr2=0x7ff7580ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7580d4910 0x7ff7580ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 -- 172.21.15.94:0/1159299369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580c9fc0 con 0x7ff75815ea30 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.458+0000 7ff757200640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b990 tx=0x7ff75000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.459+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750019070 con 0x7ff75815ea30 2024-05-27T12:37:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.459+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75000cba0 con 0x7ff75815ea30 2024-05-27T12:37:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.459+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750014440 con 0x7ff75815ea30 2024-05-27T12:37:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.459+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581f7f30 con 0x7ff75815ea30 2024-05-27T12:37:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.459+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581f84f0 con 0x7ff75815ea30 2024-05-27T12:37:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.460+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff758075460 con 0x7ff75815ea30 2024-05-27T12:37:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.461+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7ff75001c070 con 0x7ff75815ea30 2024-05-27T12:37:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.462+0000 7ff74f400640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 0x7ff73406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.462+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff75008bb80 con 0x7ff75815ea30 2024-05-27T12:37:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.463+0000 7ff756800640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 0x7ff73406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.464+0000 7ff756800640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 0x7ff73406ef90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff748034000 tx=0x7ff74802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.467+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff750059d20 con 0x7ff75815ea30 2024-05-27T12:37:42.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.745+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff75806bb30 con 0x7ff75815ea30 2024-05-27T12:37:42.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.746+0000 7ff74f400640 1 -- 172.21.15.94:0/1159299369 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1830 (secure 0 0 0) 0x7ff750059620 con 0x7ff75815ea30 2024-05-27T12:37:42.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:42.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:42.748 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:42.749 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 183, 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:42.750 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:42.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:42.753 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:42.754 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2447, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 10159721714, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 20204732416, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 752889380864, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 130681862, 2024-05-27T12:37:42.755 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 32 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:42.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:42.757 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:42.758 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": {} 2024-05-27T12:37:42.759 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:42.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 msgr2=0x7ff73406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 0x7ff73406ef90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff748034000 tx=0x7ff74802e000 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 msgr2=0x7ff7580b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b990 tx=0x7ff75000be50 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 shutdown_connections 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7ff73406cad0 0x7ff73406ef90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7580d4910 0x7ff7580ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff75815ea30 0x7ff7580b3130 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 --2- 172.21.15.94:0/1159299369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7580b4510 0x7ff7581f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 >> 172.21.15.94:0/1159299369 conn(0x7ff75809da70 msgr2=0x7ff75806e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 shutdown_connections 2024-05-27T12:37:42.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:42.749+0000 7ff75ca00640 1 -- 172.21.15.94:0/1159299369 wait complete. 2024-05-27T12:37:44.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.013+0000 7efd00400640 1 Processor -- start 2024-05-27T12:37:45.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.013+0000 7efd00400640 1 -- start start 2024-05-27T12:37:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.015+0000 7efd00400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efcf8071f20 0x7efcf8072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.015+0000 7efd00400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.015+0000 7efd00400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf80ce200 con 0x7efcf8144bb0 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.015+0000 7efd00400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7efcf8144f90 con 0x7efcf8071f20 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.015+0000 7efd00400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7efcf80728c0 con 0x7efcf80cf040 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efcf8071f20 0x7efcf8072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfe400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efcf8071f20 0x7efcf8072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59872/0 (socket says 172.21.15.94:59872) 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 -- 172.21.15.94:0/2886153652 learned_addr learned my addr 172.21.15.94:0/2886153652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:45.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfe400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:35284/0 (socket says 172.21.15.94:35284) 2024-05-27T12:37:45.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 -- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 msgr2=0x7efcf80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:45.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 --2- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 -- 172.21.15.94:0/2886153652 >> v1:172.21.15.97:6789/0 conn(0x7efcf8144bb0 legacy=0x7efcf80cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:45.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.016+0000 7efcfd000640 1 -- 172.21.15.94:0/2886153652 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf80cdc40 con 0x7efcf8071f20 2024-05-27T12:37:45.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.018+0000 7efcfd000640 1 --2- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf8072380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efce400bb10 tx=0x7efce402c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f522cbcdaa669e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.018+0000 7efcf7e00640 1 -- 172.21.15.94:0/2886153652 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4032790 con 0x7efcf8071f20 2024-05-27T12:37:45.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.018+0000 7efcf7e00640 1 -- 172.21.15.94:0/2886153652 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce4038070 con 0x7efcf8071f20 2024-05-27T12:37:45.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.018+0000 7efcf7e00640 1 -- 172.21.15.94:0/2886153652 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4032b50 con 0x7efcf8071f20 2024-05-27T12:37:45.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.019+0000 7efd00400640 1 -- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 msgr2=0x7efcf8072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.019+0000 7efd00400640 1 --2- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf8072380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efce400bb10 tx=0x7efce402c8d0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 -- 172.21.15.94:0/2886153652 shutdown_connections 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 --2- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf80cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 --2- 172.21.15.94:0/2886153652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf8072380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 -- 172.21.15.94:0/2886153652 >> 172.21.15.94:0/2886153652 conn(0x7efcf80b3830 msgr2=0x7efcf80b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 -- 172.21.15.94:0/2886153652 shutdown_connections 2024-05-27T12:37:45.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.020+0000 7efd00400640 1 -- 172.21.15.94:0/2886153652 wait complete. 2024-05-27T12:37:45.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.022+0000 7efd00400640 1 Processor -- start 2024-05-27T12:37:45.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.022+0000 7efd00400640 1 -- start start 2024-05-27T12:37:45.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf813d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf81e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcf80ceb20 con 0x7efcf8144bb0 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efcf80ce840 con 0x7efcf8071f20 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.023+0000 7efd00400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7efcf8145330 con 0x7efcf80cf040 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56858/0 (socket says 172.21.15.94:56858) 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 -- 172.21.15.94:0/3366778880 learned_addr learned my addr 172.21.15.94:0/3366778880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfd000640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf81e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfda00640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf813d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 -- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 msgr2=0x7efcf81e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf81e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 -- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 msgr2=0x7efcf813d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf813d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.024+0000 7efcfe400640 1 -- 172.21.15.94:0/3366778880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf813ecc0 con 0x7efcf8144bb0 2024-05-27T12:37:45.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efcfe400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efcf0007e40 tx=0x7efcf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0014620 con 0x7efcf8144bb0 2024-05-27T12:37:45.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf0019070 con 0x7efcf8144bb0 2024-05-27T12:37:45.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf813dde0 con 0x7efcf8144bb0 2024-05-27T12:37:45.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0014a20 con 0x7efcf8144bb0 2024-05-27T12:37:45.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.025+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf813f210 con 0x7efcf8144bb0 2024-05-27T12:37:45.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.026+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8075460 con 0x7efcf8144bb0 2024-05-27T12:37:45.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.027+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7efcf0014bc0 con 0x7efcf8144bb0 2024-05-27T12:37:45.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.028+0000 7efcf5600640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 0x7efcd806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.028+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efcf008c760 con 0x7efcf8144bb0 2024-05-27T12:37:45.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.029+0000 7efcfda00640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 0x7efcd806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.030+0000 7efcfda00640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 0x7efcd806ef60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcec006fd0 tx=0x7efcec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.036+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcf005a870 con 0x7efcf8144bb0 2024-05-27T12:37:45.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.504+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efcf806e680 con 0x7efcf8144bb0 2024-05-27T12:37:45.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.506+0000 7efcf5600640 1 -- 172.21.15.94:0/3366778880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2095 (secure 0 0 0) 0x7efcf005a170 con 0x7efcf8144bb0 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:45.507 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:45.508 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 185, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:45.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2590, 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 10738614377, 2024-05-27T12:37:45.510 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 21744226304, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 751349886976, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "read_bytes_sec": 170, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 89455354, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 24 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:45.512 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.513 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:45.514 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:45.514 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:45.514 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:45.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "1 ongoing clones; Avg progress is 0.0% (1s)\n [............................] ", 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 0, 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:45.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 msgr2=0x7efcd806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 0x7efcd806ef60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcec006fd0 tx=0x7efcec008040 comp rx=0 tx=0).stop 2024-05-27T12:37:45.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 msgr2=0x7efcf81e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efcf0007e40 tx=0x7efcf000e4b0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 shutdown_connections 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7efcd806caa0 0x7efcd806ef60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcf8144bb0 0x7efcf81e4620 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efcf80cf040 0x7efcf81e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 --2- 172.21.15.94:0/3366778880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcf8071f20 0x7efcf813d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.511+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 >> 172.21.15.94:0/3366778880 conn(0x7efcf80b3830 msgr2=0x7efcf80b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.512+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 shutdown_connections 2024-05-27T12:37:45.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.512+0000 7efd00400640 1 -- 172.21.15.94:0/3366778880 wait complete. 2024-05-27T12:37:45.539 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-27T12:37:45.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.801+0000 7eff50c00640 1 Processor -- start 2024-05-27T12:37:45.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.801+0000 7eff50c00640 1 -- start start 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c15e930 0x7eff4c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7eff4c0adb10 0x7eff4c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 0x7eff4c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7eff4c072930 con 0x7eff4c0adb10 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7eff4c15ed10 con 0x7eff4c06fd20 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.804+0000 7eff50c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c072000 con 0x7eff4c15e930 2024-05-27T12:37:45.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.805+0000 7eff49400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7eff4c0adb10 0x7eff4c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.805+0000 7eff49400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7eff4c0adb10 0x7eff4c0b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56860/0 (socket says 172.21.15.94:56860) 2024-05-27T12:37:45.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.805+0000 7eff49400640 1 -- 172.21.15.94:0/3218090743 learned_addr learned my addr 172.21.15.94:0/3218090743 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:45.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff4a800640 1 --2- 172.21.15.94:0/3218090743 >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 0x7eff4c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff49400640 1 -- 172.21.15.94:0/3218090743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c15e930 msgr2=0x7eff4c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff49400640 1 --2- 172.21.15.94:0/3218090743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c15e930 0x7eff4c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff49400640 1 -- 172.21.15.94:0/3218090743 >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 msgr2=0x7eff4c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff49400640 1 --2- 172.21.15.94:0/3218090743 >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 0x7eff4c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff49400640 1 -- 172.21.15.94:0/3218090743 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c0722e0 con 0x7eff4c0adb10 2024-05-27T12:37:45.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.806+0000 7eff4a800640 1 --2- 172.21.15.94:0/3218090743 >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 0x7eff4c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:45.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.807+0000 7eff49400640 1 --2- 172.21.15.94:0/3218090743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c0adb10 0x7eff4c0b4540 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff4c072210 tx=0x7eff3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d03194e98d92da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.811+0000 7eff48a00640 1 -- 172.21.15.94:0/3218090743 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38037070 con 0x7eff4c0adb10 2024-05-27T12:37:45.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.811+0000 7eff48a00640 1 -- 172.21.15.94:0/3218090743 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff38032440 con 0x7eff4c0adb10 2024-05-27T12:37:45.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.811+0000 7eff48a00640 1 -- 172.21.15.94:0/3218090743 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff380327f0 con 0x7eff4c0adb10 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 -- 172.21.15.94:0/3218090743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c0adb10 msgr2=0x7eff4c0b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 --2- 172.21.15.94:0/3218090743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c0adb10 0x7eff4c0b4540 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff4c072210 tx=0x7eff3802a1e0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 -- 172.21.15.94:0/3218090743 shutdown_connections 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 --2- 172.21.15.94:0/3218090743 >> v2:172.21.15.97:3300/0 conn(0x7eff4c06fd20 0x7eff4c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 --2- 172.21.15.94:0/3218090743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c0adb10 0x7eff4c0b4540 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 --2- 172.21.15.94:0/3218090743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c15e930 0x7eff4c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.812+0000 7eff50c00640 1 -- 172.21.15.94:0/3218090743 >> 172.21.15.94:0/3218090743 conn(0x7eff4c0b2240 msgr2=0x7eff4c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.813+0000 7eff50c00640 1 -- 172.21.15.94:0/3218090743 shutdown_connections 2024-05-27T12:37:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.813+0000 7eff50c00640 1 -- 172.21.15.94:0/3218090743 wait complete. 2024-05-27T12:37:45.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.815+0000 7eff50c00640 1 Processor -- start 2024-05-27T12:37:45.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.816+0000 7eff50c00640 1 -- start start 2024-05-27T12:37:45.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15e930 0x7eff4c141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0acc80 0x7eff4c0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c0b3850 con 0x7eff4c15e930 2024-05-27T12:37:45.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c0b36d0 con 0x7eff4c06fd20 2024-05-27T12:37:45.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.819+0000 7eff50c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c0afbf0 con 0x7eff4c0acc80 2024-05-27T12:37:45.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.820+0000 7eff49e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.820+0000 7eff49e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:59892/0 (socket says 172.21.15.94:59892) 2024-05-27T12:37:45.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.820+0000 7eff49e00640 1 -- 172.21.15.94:0/784925130 learned_addr learned my addr 172.21.15.94:0/784925130 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.820+0000 7eff4a800640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0acc80 0x7eff4c0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.821+0000 7eff49e00640 1 -- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0acc80 msgr2=0x7eff4c0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.821+0000 7eff49e00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0acc80 0x7eff4c0a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.821+0000 7eff49e00640 1 -- 172.21.15.94:0/784925130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15e930 msgr2=0x7eff4c141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.821+0000 7eff49e00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15e930 0x7eff4c141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.821+0000 7eff49e00640 1 -- 172.21.15.94:0/784925130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c0ad4f0 con 0x7eff4c06fd20 2024-05-27T12:37:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.822+0000 7eff49e00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00b440 tx=0x7eff3c00b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.823+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c015020 con 0x7eff4c06fd20 2024-05-27T12:37:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.823+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c0ce870 con 0x7eff4c06fd20 2024-05-27T12:37:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.823+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c0cee30 con 0x7eff4c06fd20 2024-05-27T12:37:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.823+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c011890 con 0x7eff4c06fd20 2024-05-27T12:37:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.823+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c015020 con 0x7eff4c06fd20 2024-05-27T12:37:45.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.828+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff24005290 con 0x7eff4c06fd20 2024-05-27T12:37:45.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.829+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7eff3c018020 con 0x7eff4c06fd20 2024-05-27T12:37:45.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.832+0000 7eff42000640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 0x7eff2c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.832+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7eff3c08b780 con 0x7eff4c06fd20 2024-05-27T12:37:45.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.833+0000 7eff49400640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 0x7eff2c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:45.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.837+0000 7eff49400640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 0x7eff2c06f190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff38034000 tx=0x7eff3802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:45.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:45.841+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff3c01f030 con 0x7eff4c06fd20 2024-05-27T12:37:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.188+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7eff24000bc0 con 0x7eff2c06ccd0 2024-05-27T12:37:46.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.330+0000 7eff42000640 1 -- 172.21.15.94:0/784925130 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff24000bc0 con 0x7eff2c06ccd0 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 msgr2=0x7eff2c06f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 0x7eff2c06f190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff38034000 tx=0x7eff3802c000 comp rx=0 tx=0).stop 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 msgr2=0x7eff4c0cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00b440 tx=0x7eff3c00b900 comp rx=0 tx=0).stop 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 shutdown_connections 2024-05-27T12:37:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7eff2c06ccd0 0x7eff2c06f190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff4c0acc80 0x7eff4c0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff4c15e930 0x7eff4c141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 --2- 172.21.15.94:0/784925130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff4c06fd20 0x7eff4c0cdb10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.332+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 >> 172.21.15.94:0/784925130 conn(0x7eff4c0b2240 msgr2=0x7eff4c0ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.333+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 shutdown_connections 2024-05-27T12:37:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.333+0000 7eff50c00640 1 -- 172.21.15.94:0/784925130 wait complete. 2024-05-27T12:37:46.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-27T12:37:46.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.521+0000 7fae64a00640 1 Processor -- start 2024-05-27T12:37:46.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.521+0000 7fae64a00640 1 -- start start 2024-05-27T12:37:46.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae64a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae600d1770 0x7fae600cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae64a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae600cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae64a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae600d2a80 con 0x7fae60162170 2024-05-27T12:37:46.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae64a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fae600d3650 con 0x7fae600d1770 2024-05-27T12:37:46.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae64a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fae600d2d20 con 0x7fae600b51d0 2024-05-27T12:37:46.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae5e800640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fae60162170 0x7fae600d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51704/0 (socket says 172.21.15.94:51704) 2024-05-27T12:37:46.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae5e800640 1 -- 172.21.15.94:0/424212949 learned_addr learned my addr 172.21.15.94:0/424212949 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585089881 0 0) 0x7fae600d2a80 con 0x7fae60162170 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae44003520 con 0x7fae60162170 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.523+0000 7fae5de00640 1 --2- 172.21.15.94:0/424212949 >> v2:172.21.15.97:3300/0 conn(0x7fae600d1770 0x7fae600cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5f200640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae600cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3108581434 0 0) 0x7fae44003520 con 0x7fae60162170 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 msgr2=0x7fae600cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae600cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 >> v2:172.21.15.97:3300/0 conn(0x7fae600d1770 msgr2=0x7fae600cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 --2- 172.21.15.94:0/424212949 >> v2:172.21.15.97:3300/0 conn(0x7fae600d1770 0x7fae600cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae600d3000 con 0x7fae60162170 2024-05-27T12:37:46.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fae50001670 con 0x7fae60162170 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5f200640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae600cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 >> v1:172.21.15.94:6789/0 conn(0x7fae60162170 legacy=0x7fae600d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae44004800 0x7fae44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae600d3650 con 0x7fae440039b0 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae600d2d20 con 0x7fae44004800 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.524+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fae4400bb20 con 0x7fae600b51d0 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5de00640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae44004800 0x7fae44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5f200640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 -- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 msgr2=0x7fae440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600b51d0 0x7fae440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 -- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae44004800 msgr2=0x7fae44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae44004800 0x7fae44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.525+0000 7fae5e800640 1 -- 172.21.15.94:0/424212949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae44003520 con 0x7fae440039b0 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.526+0000 7fae5e800640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae44003dd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae50007fd0 tx=0x7fae44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51887b59e894103f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.526+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50007d50 con 0x7fae440039b0 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.526+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae50005af0 con 0x7fae440039b0 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.526+0000 7fae5d400640 1 -- 172.21.15.94:0/424212949 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50007070 con 0x7fae440039b0 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 -- 172.21.15.94:0/424212949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 msgr2=0x7fae44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 --2- 172.21.15.94:0/424212949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae44003dd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae50007fd0 tx=0x7fae44005830 comp rx=0 tx=0).stop 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae5f200640 1 -- 172.21.15.94:0/424212949 reap_dead start 2024-05-27T12:37:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 -- 172.21.15.94:0/424212949 shutdown_connections 2024-05-27T12:37:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 -- 172.21.15.94:0/424212949 >> 172.21.15.94:0/424212949 conn(0x7fae6008bfc0 msgr2=0x7fae600d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 -- 172.21.15.94:0/424212949 shutdown_connections 2024-05-27T12:37:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.527+0000 7fae64a00640 1 -- 172.21.15.94:0/424212949 wait complete. 2024-05-27T12:37:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.528+0000 7fae64a00640 1 Processor -- start 2024-05-27T12:37:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.529+0000 7fae64a00640 1 -- start start 2024-05-27T12:37:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600ce340 0x7fae600b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6015e7a0 0x7fae600b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae600b3660 con 0x7fae440039b0 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae600b34e0 con 0x7fae6015e7a0 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae64a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fae600b37e0 con 0x7fae600ce340 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56894/0 (socket says 172.21.15.94:56894) 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 -- 172.21.15.94:0/3686361866 learned_addr learned my addr 172.21.15.94:0/3686361866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 -- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600ce340 msgr2=0x7fae600b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600ce340 0x7fae600b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 -- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6015e7a0 msgr2=0x7fae600b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6015e7a0 0x7fae600b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 -- 172.21.15.94:0/3686361866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae600cd2a0 con 0x7fae440039b0 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.530+0000 7fae5e800640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c300 tx=0x7fae50001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.531+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500049f0 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.531+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae50004b90 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.531+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50007070 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.531+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae600cd580 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.531+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae600cdb40 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.532+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae60075460 con 0x7fae440039b0 2024-05-27T12:37:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.533+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fae50045020 con 0x7fae440039b0 2024-05-27T12:37:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.534+0000 7fae56a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 0x7fae4006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.534+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fae500abde0 con 0x7fae440039b0 2024-05-27T12:37:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.534+0000 7fae5de00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 0x7fae4006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.535+0000 7fae5de00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 0x7fae4006ef60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006a90 tx=0x7fae4c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.538+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae5004c020 con 0x7fae440039b0 2024-05-27T12:37:46.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.749+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fae600d3a40 con 0x7fae4006caa0 2024-05-27T12:37:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.753+0000 7fae56a00640 1 -- 172.21.15.94:0/3686361866 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fae4c00b5c0 con 0x7fae4006caa0 2024-05-27T12:37:46.752 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:37:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-05-27T12:37:46.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:46.753 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 msgr2=0x7fae4006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 0x7fae4006ef60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006a90 tx=0x7fae4c006a20 comp rx=0 tx=0).stop 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 msgr2=0x7fae600ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c300 tx=0x7fae50001a40 comp rx=0 tx=0).stop 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 shutdown_connections 2024-05-27T12:37:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fae4006caa0 0x7fae4006ef60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6015e7a0 0x7fae600b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae600ce340 0x7fae600b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 --2- 172.21.15.94:0/3686361866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae440039b0 0x7fae600ccb30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.755+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 >> 172.21.15.94:0/3686361866 conn(0x7fae6008bfc0 msgr2=0x7fae600cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.756+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 shutdown_connections 2024-05-27T12:37:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.756+0000 7fae64a00640 1 -- 172.21.15.94:0/3686361866 wait complete. 2024-05-27T12:37:46.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.961+0000 7f3985e00640 1 Processor -- start 2024-05-27T12:37:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.961+0000 7f3985e00640 1 -- start start 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.962+0000 7f3985e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3980072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.962+0000 7f3985e00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398006f860 con 0x7f398006f1f0 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.962+0000 7f3985e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39800cecb0 con 0x7f398015ac60 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.962+0000 7f3985e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f39800cf300 con 0x7f39800725f0 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f398006f1f0 0x7f39800cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:51710/0 (socket says 172.21.15.94:51710) 2024-05-27T12:37:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397fe00640 1 -- 172.21.15.94:0/2081911770 learned_addr learned my addr 172.21.15.94:0/2081911770 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:37:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364065757 0 0) 0x7f398006f860 con 0x7f398006f1f0 2024-05-27T12:37:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3964005730 con 0x7f398006f1f0 2024-05-27T12:37:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f3985400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3980072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 984260053 0 0) 0x7f3964005730 con 0x7f398006f1f0 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 msgr2=0x7f3980072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3980072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 >> v1:172.21.15.97:6789/0 conn(0x7f398015ac60 legacy=0x7f39800b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800cef90 con 0x7f398006f1f0 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f3968001670 con 0x7f398006f1f0 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 >> v1:172.21.15.94:6789/0 conn(0x7f398006f1f0 legacy=0x7f39800cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.963+0000 7f397f400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3964005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397f400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006200 0x7f3964006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397f400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964006f50 0x7f39640073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39800cf300 con 0x7f3964006f50 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f396400dd60 con 0x7f3964006200 2024-05-27T12:37:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f396400df60 con 0x7f39800725f0 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964006f50 0x7f39640073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f397fe00640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006200 0x7f3964006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3984a00640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3964005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 -- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 msgr2=0x7f3964005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3964005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 -- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006200 msgr2=0x7f3964006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006200 0x7f3964006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3985400640 1 -- 172.21.15.94:0/2081911770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3964005730 con 0x7f3964006f50 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.964+0000 7f3984a00640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39800725f0 0x7f3964005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.965+0000 7f3985400640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964006f50 0x7f39640073d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3978007330 tx=0x7f3964007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8cb1b33f9dab1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.965+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978033070 con 0x7f3964006f50 2024-05-27T12:37:46.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.965+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397802f650 con 0x7f3964006f50 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f397f400640 1 -- 172.21.15.94:0/2081911770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978009be0 con 0x7f3964006f50 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 -- 172.21.15.94:0/2081911770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964006f50 msgr2=0x7f39640073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 --2- 172.21.15.94:0/2081911770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3964006f50 0x7f39640073d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3978007330 tx=0x7f3964007a40 comp rx=0 tx=0).stop 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985400640 1 -- 172.21.15.94:0/2081911770 reap_dead start 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 -- 172.21.15.94:0/2081911770 shutdown_connections 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 -- 172.21.15.94:0/2081911770 >> 172.21.15.94:0/2081911770 conn(0x7f39800b2160 msgr2=0x7f39800b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 -- 172.21.15.94:0/2081911770 shutdown_connections 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.966+0000 7f3985e00640 1 -- 172.21.15.94:0/2081911770 wait complete. 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.968+0000 7f3985e00640 1 Processor -- start 2024-05-27T12:37:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.968+0000 7f3985e00640 1 -- start start 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006f50 0x7f398014a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f398014ad30 0x7f398015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3980072a80 con 0x7f39801e3e50 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39800ce960 con 0x7f3964006f50 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985e00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f3980072c00 con 0x7f398014ad30 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56924/0 (socket says 172.21.15.94:56924) 2024-05-27T12:37:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 -- 172.21.15.94:0/2238055957 learned_addr learned my addr 172.21.15.94:0/2238055957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f397fe00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f398014ad30 0x7f398015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3984a00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006f50 0x7f398014a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 -- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f398014ad30 msgr2=0x7f398015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f398014ad30 0x7f398015bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 -- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006f50 msgr2=0x7f398014a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006f50 0x7f398014a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.969+0000 7f3985400640 1 -- 172.21.15.94:0/2238055957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398015c810 con 0x7f39801e3e50 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f3985400640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f397802b030 tx=0x7f3978004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978033040 con 0x7f39801e3e50 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397802f450 con 0x7f39801e3e50 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978009be0 con 0x7f39801e3e50 2024-05-27T12:37:46.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398015caf0 con 0x7f39801e3e50 2024-05-27T12:37:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.970+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398015d0b0 con 0x7f39801e3e50 2024-05-27T12:37:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.971+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3980075460 con 0x7f39801e3e50 2024-05-27T12:37:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.972+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f3978036070 con 0x7f39801e3e50 2024-05-27T12:37:46.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.973+0000 7f397cc00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 0x7f395c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:46.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.973+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f39780ac520 con 0x7f39801e3e50 2024-05-27T12:37:46.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.974+0000 7f3984a00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 0x7f395c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:46.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.975+0000 7f3984a00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 0x7f395c06ef90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39801543f0 tx=0x7f3970009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:46.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:46.977+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f397807a630 con 0x7f39801e3e50 2024-05-27T12:37:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.230+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f39800b0f30 con 0x7f39801e3e50 2024-05-27T12:37:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.231+0000 7f397cc00640 1 -- 172.21.15.94:0/2238055957 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2121 (secure 0 0 0) 0x7f3978079f30 con 0x7f39801e3e50 2024-05-27T12:37:47.230 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:47.230 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:47.231 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 187, 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:47.232 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:47.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2772, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 11506280656, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 23334150144, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 749759963136, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:47.234 INFO:teuthology.orchestra.run.smithi094.stdout: "read_bytes_sec": 61846065, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 153411420, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 15, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 42 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:47.235 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:47.236 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 30.0 seconds before exiting (3s)\n [============================] ", 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:37:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 msgr2=0x7f395c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 0x7f395c06ef90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39801543f0 tx=0x7f3970009040 comp rx=0 tx=0).stop 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 msgr2=0x7f39801e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f397802b030 tx=0x7f3978004ab0 comp rx=0 tx=0).stop 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 shutdown_connections 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f395c06cad0 0x7f395c06ef90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39801e3e50 0x7f39801e42e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f398014ad30 0x7f398015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 --2- 172.21.15.94:0/2238055957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3964006f50 0x7f398014a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 >> 172.21.15.94:0/2238055957 conn(0x7f39800b2160 msgr2=0x7f3980157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 shutdown_connections 2024-05-27T12:37:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:47.233+0000 7f3985e00640 1 -- 172.21.15.94:0/2238055957 wait complete. 2024-05-27T12:37:52.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:52.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.462+0000 7f293b200640 1 Processor -- start 2024-05-27T12:37:52.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.463+0000 7f293b200640 1 -- start start 2024-05-27T12:37:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.464+0000 7f293b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f293c06f5e0 0x7f293c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.464+0000 7f293b200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f293c13e9e0 con 0x7f293c13e600 2024-05-27T12:37:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.464+0000 7f293b200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f293c15ba50 con 0x7f293c0cd480 2024-05-27T12:37:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.464+0000 7f293b200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f293c15bd10 con 0x7f293c06f5e0 2024-05-27T12:37:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.465+0000 7f2939e00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f293c13e600 0x7f293c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:32898/0 (socket says 172.21.15.94:32898) 2024-05-27T12:37:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.465+0000 7f2939e00640 1 -- 172.21.15.94:0/2396041630 learned_addr learned my addr 172.21.15.94:0/2396041630 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:37:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.465+0000 7f2939400640 1 --2- 172.21.15.94:0/2396041630 >> v2:172.21.15.97:3300/0 conn(0x7f293c06f5e0 0x7f293c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.465+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035550611 0 0) 0x7f292c001110 con 0x7f293c13e600 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.465+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f291c005730 con 0x7f293c13e600 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 701775352 0 0) 0x7f291c005730 con 0x7f293c13e600 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 >> v2:172.21.15.97:3300/0 conn(0x7f293c06f5e0 msgr2=0x7f293c071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 --2- 172.21.15.94:0/2396041630 >> v2:172.21.15.97:3300/0 conn(0x7f293c06f5e0 0x7f293c071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 >> v1:172.21.15.97:6789/0 conn(0x7f293c0cd480 legacy=0x7f293c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c15bff0 con 0x7f293c13e600 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f292c001840 con 0x7f293c13e600 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 >> v1:172.21.15.94:6789/0 conn(0x7f293c13e600 legacy=0x7f293c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 0x7f291c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c006570 0x7f291c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f291c007180 0x7f291c00ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f293c15bd10 con 0x7f291c005c10 2024-05-27T12:37:52.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f291c010510 con 0x7f291c006570 2024-05-27T12:37:52.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f291c0106c0 con 0x7f291c007180 2024-05-27T12:37:52.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 0x7f291c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939400640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c006570 0x7f291c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 -- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f291c007180 msgr2=0x7f291c00ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f291c007180 0x7f291c00ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 -- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c006570 msgr2=0x7f291c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c006570 0x7f291c006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f2939e00640 1 -- 172.21.15.94:0/2396041630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c005730 con 0x7f291c005c10 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.466+0000 7f293a800640 1 -- 172.21.15.94:0/2396041630 reap_dead start 2024-05-27T12:37:52.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.467+0000 7f2939e00640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 0x7f291c006030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f292c007fd0 tx=0x7f291c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc30c7f09c6d9104 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.468+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c007e30 con 0x7f291c005c10 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.468+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c005af0 con 0x7f291c005c10 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.468+0000 7f2938a00640 1 -- 172.21.15.94:0/2396041630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c007070 con 0x7f291c005c10 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.468+0000 7f293b200640 1 -- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 msgr2=0x7f291c006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.468+0000 7f293b200640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 0x7f291c006030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f292c007fd0 tx=0x7f291c007a40 comp rx=0 tx=0).stop 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.469+0000 7f293b200640 1 -- 172.21.15.94:0/2396041630 shutdown_connections 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.469+0000 7f293b200640 1 --2- 172.21.15.94:0/2396041630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f291c005c10 0x7f291c006030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.469+0000 7f293b200640 1 -- 172.21.15.94:0/2396041630 >> 172.21.15.94:0/2396041630 conn(0x7f293c0b2160 msgr2=0x7f293c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.469+0000 7f293b200640 1 -- 172.21.15.94:0/2396041630 shutdown_connections 2024-05-27T12:37:52.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.469+0000 7f293b200640 1 -- 172.21.15.94:0/2396041630 wait complete. 2024-05-27T12:37:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.470+0000 7f293b200640 1 Processor -- start 2024-05-27T12:37:52.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 -- start start 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 0x7f293c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f293c159310 0x7f293c0ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f293c15c920 con 0x7f293c0a9490 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f293c15c5b0 con 0x7f291c005c10 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.471+0000 7f293b200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f293c071d60 con 0x7f293c159310 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56892/0 (socket says 172.21.15.94:56892) 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 -- 172.21.15.94:0/3258170883 learned_addr learned my addr 172.21.15.94:0/3258170883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f2939e00640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 0x7f293c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 -- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f293c159310 msgr2=0x7f293c0ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f2939400640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f293c159310 0x7f293c0ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f293c159310 0x7f293c0ab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 -- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 msgr2=0x7f293c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 0x7f293c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 -- 172.21.15.94:0/3258170883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c0ae3e0 con 0x7f293c0a9490 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f2939e00640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 0x7f293c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.472+0000 7f293a800640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f293400c390 tx=0x7f293400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.473+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934007480 con 0x7f293c0a9490 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.473+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2934004020 con 0x7f293c0a9490 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.473+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c0ae6c0 con 0x7f293c0a9490 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.473+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934007640 con 0x7f293c0a9490 2024-05-27T12:37:52.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.473+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1e3c80 con 0x7f293c0a9490 2024-05-27T12:37:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.474+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c075460 con 0x7f293c0a9490 2024-05-27T12:37:52.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.475+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f2934011070 con 0x7f293c0a9490 2024-05-27T12:37:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.477+0000 7f2932000640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 0x7f291806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:52.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.477+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f293408af10 con 0x7f293c0a9490 2024-05-27T12:37:52.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.477+0000 7f2939e00640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 0x7f291806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:52.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.478+0000 7f2939e00640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 0x7f291806ef60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f292c033000 tx=0x7f292c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:52.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.482+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29340590b0 con 0x7f293c0a9490 2024-05-27T12:37:52.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.760+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f293c1e3e50 con 0x7f293c0a9490 2024-05-27T12:37:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.761+0000 7f2932000640 1 -- 172.21.15.94:0/3258170883 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2122 (secure 0 0 0) 0x7f29340589b0 con 0x7f293c0a9490 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 193, 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2896, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013792172, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24231743488, 2024-05-27T12:37:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748862369792, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "read_bytes_sec": 107609632, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 154443913, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 26, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 42 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (7s)\n [============================] ", 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:52.768 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 msgr2=0x7f291806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 0x7f291806ef60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f292c033000 tx=0x7f292c02e000 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 msgr2=0x7f293c0a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f293400c390 tx=0x7f293400c850 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 shutdown_connections 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f291806caa0 0x7f291806ef60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f293c0a9490 0x7f293c0a9910 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f293c159310 0x7f293c0ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 --2- 172.21.15.94:0/3258170883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f291c005c10 0x7f293c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 >> 172.21.15.94:0/3258170883 conn(0x7f293c0b2160 msgr2=0x7f293c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 shutdown_connections 2024-05-27T12:37:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:52.764+0000 7f293b200640 1 -- 172.21.15.94:0/3258170883 wait complete. 2024-05-27T12:37:57.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:37:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.990+0000 7fe6b4000640 1 Processor -- start 2024-05-27T12:37:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.991+0000 7fe6b4000640 1 -- start start 2024-05-27T12:37:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6ac15e920 0x7fe6ac162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0b2f10 0x7fe6ac071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe6ac072460 0x7fe6ac06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac072890 con 0x7fe6ac072460 2024-05-27T12:37:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac15ed00 con 0x7fe6ac15e920 2024-05-27T12:37:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.992+0000 7fe6b4000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac0cd580 con 0x7fe6ac0b2f10 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe6ac072460 0x7fe6ac06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe6ac072460 0x7fe6ac06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56908/0 (socket says 172.21.15.94:56908) 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 -- 172.21.15.94:0/1452797413 learned_addr learned my addr 172.21.15.94:0/1452797413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b0c00640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0b2f10 0x7fe6ac071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b1600640 1 --2- 172.21.15.94:0/1452797413 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac15e920 0x7fe6ac162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 -- 172.21.15.94:0/1452797413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0b2f10 msgr2=0x7fe6ac071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0b2f10 0x7fe6ac071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 -- 172.21.15.94:0/1452797413 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac15e920 msgr2=0x7fe6ac162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1452797413 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac15e920 0x7fe6ac162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.993+0000 7fe6b2000640 1 -- 172.21.15.94:0/1452797413 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac0cd860 con 0x7fe6ac072460 2024-05-27T12:37:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac072460 0x7fe6ac06df20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac070070 tx=0x7fe6a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52466f6dd362daca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6abe00640 1 -- 172.21.15.94:0/1452797413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4036070 con 0x7fe6ac072460 2024-05-27T12:37:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6abe00640 1 -- 172.21.15.94:0/1452797413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a4031600 con 0x7fe6ac072460 2024-05-27T12:37:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6abe00640 1 -- 172.21.15.94:0/1452797413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40319d0 con 0x7fe6ac072460 2024-05-27T12:37:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6b4000640 1 -- 172.21.15.94:0/1452797413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac072460 msgr2=0x7fe6ac06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.995+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac072460 0x7fe6ac06df20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac070070 tx=0x7fe6a402a6a0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 -- 172.21.15.94:0/1452797413 shutdown_connections 2024-05-27T12:37:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac072460 0x7fe6ac06df20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1452797413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0b2f10 0x7fe6ac071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1452797413 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac15e920 0x7fe6ac162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 -- 172.21.15.94:0/1452797413 >> 172.21.15.94:0/1452797413 conn(0x7fe6ac0b3830 msgr2=0x7fe6ac0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 -- 172.21.15.94:0/1452797413 shutdown_connections 2024-05-27T12:37:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.996+0000 7fe6b4000640 1 -- 172.21.15.94:0/1452797413 wait complete. 2024-05-27T12:37:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.997+0000 7fe6b4000640 1 Processor -- start 2024-05-27T12:37:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.998+0000 7fe6b4000640 1 -- start start 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.998+0000 7fe6b4000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac072460 0x7fe6ac0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b4000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0b2f10 0x7fe6ac145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0ce100 con 0x7fe6ac15e920 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b4000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0cde20 con 0x7fe6ac0b2f10 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b4000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac163380 con 0x7fe6ac072460 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56920/0 (socket says 172.21.15.94:56920) 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 -- 172.21.15.94:0/1670575918 learned_addr learned my addr 172.21.15.94:0/1670575918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b1600640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac072460 0x7fe6ac0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b0c00640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0b2f10 0x7fe6ac145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:57.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 -- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac072460 msgr2=0x7fe6ac0ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac072460 0x7fe6ac0ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 -- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0b2f10 msgr2=0x7fe6ac145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0b2f10 0x7fe6ac145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:57.999+0000 7fe6b2000640 1 -- 172.21.15.94:0/1670575918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac146220 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6b2000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4034040 tx=0x7fe6a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4036050 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a4042070 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4009510 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac146500 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.000+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac143c30 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.002+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ac1466d0 con 0x7fe6ac15e920 2024-05-27T12:37:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.002+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe6a4007bd0 con 0x7fe6ac15e920 2024-05-27T12:37:58.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.003+0000 7fe6a9600640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 0x7fe68c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:37:58.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.003+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe6a40acde0 con 0x7fe6ac15e920 2024-05-27T12:37:58.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.004+0000 7fe6b1600640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 0x7fe68c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:37:58.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.005+0000 7fe6b1600640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 0x7fe68c06ef60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:37:58.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.008+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6a407af10 con 0x7fe6ac15e920 2024-05-27T12:37:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.282+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe6ac075460 con 0x7fe6ac15e920 2024-05-27T12:37:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.283+0000 7fe6a9600640 1 -- 172.21.15.94:0/1670575918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2123 (secure 0 0 0) 0x7fe6a407a810 con 0x7fe6ac15e920 2024-05-27T12:37:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:37:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 198, 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:37:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:37:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2893, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013837553, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24298930176, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748795183104, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "read_bytes_sec": 107612366, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 106229058, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 26, 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 31 2024-05-27T12:37:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:37:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (13s)\n [============================] ", 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:37:58.289 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:37:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 msgr2=0x7fe68c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 0x7fe68c06ef60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0006fd0 tx=0x7fe6a0008040 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 msgr2=0x7fe6ac142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4034040 tx=0x7fe6a4004a40 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 shutdown_connections 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe68c06caa0 0x7fe68c06ef60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0b2f10 0x7fe6ac145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6ac15e920 0x7fe6ac142b60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 --2- 172.21.15.94:0/1670575918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac072460 0x7fe6ac0ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 >> 172.21.15.94:0/1670575918 conn(0x7fe6ac0b3830 msgr2=0x7fe6ac0cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 shutdown_connections 2024-05-27T12:37:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:37:58.286+0000 7fe6b4000640 1 -- 172.21.15.94:0/1670575918 wait complete. 2024-05-27T12:37:59.258 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:37:59.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:38:03.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:38:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.516+0000 7f5fcf200640 1 Processor -- start 2024-05-27T12:38:03.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.516+0000 7f5fcf200640 1 -- start start 2024-05-27T12:38:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.518+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.518+0000 7f5fcf200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fd01460d0 0x7f5fd00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.518+0000 7f5fcf200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd00ceae0 con 0x7f5fd00ce610 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.518+0000 7f5fcf200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd01602c0 con 0x7f5fd01460d0 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.518+0000 7f5fcf200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd01605a0 con 0x7f5fd0145cf0 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.519+0000 7f5fcde00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.519+0000 7f5fc5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fd01460d0 0x7f5fd00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.519+0000 7f5fcde00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:43376/0 (socket says 172.21.15.94:43376) 2024-05-27T12:38:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.519+0000 7f5fcde00640 1 -- 172.21.15.94:0/4193418251 learned_addr learned my addr 172.21.15.94:0/4193418251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.520+0000 7f5fcde00640 1 -- 172.21.15.94:0/4193418251 >> v2:172.21.15.97:3300/0 conn(0x7f5fd01460d0 msgr2=0x7f5fd00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.520+0000 7f5fcde00640 1 --2- 172.21.15.94:0/4193418251 >> v2:172.21.15.97:3300/0 conn(0x7f5fd01460d0 0x7f5fd00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.520+0000 7f5fcde00640 1 -- 172.21.15.94:0/4193418251 >> v1:172.21.15.97:6789/0 conn(0x7f5fd00ce610 legacy=0x7f5fd0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.520+0000 7f5fcde00640 1 -- 172.21.15.94:0/4193418251 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd0160880 con 0x7f5fd0145cf0 2024-05-27T12:38:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.521+0000 7f5fcde00640 1 --2- 172.21.15.94:0/4193418251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc6b54ec038977a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.521+0000 7f5fcd400640 1 -- 172.21.15.94:0/4193418251 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0037070 con 0x7f5fd0145cf0 2024-05-27T12:38:03.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.521+0000 7f5fcd400640 1 -- 172.21.15.94:0/4193418251 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc0032440 con 0x7f5fd0145cf0 2024-05-27T12:38:03.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.521+0000 7f5fcd400640 1 -- 172.21.15.94:0/4193418251 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0032810 con 0x7f5fd0145cf0 2024-05-27T12:38:03.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 -- 172.21.15.94:0/4193418251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 msgr2=0x7f5fd0160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 --2- 172.21.15.94:0/4193418251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002a270 comp rx=0 tx=0).stop 2024-05-27T12:38:03.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 -- 172.21.15.94:0/4193418251 shutdown_connections 2024-05-27T12:38:03.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 --2- 172.21.15.94:0/4193418251 >> v2:172.21.15.97:3300/0 conn(0x7f5fd01460d0 0x7f5fd00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 --2- 172.21.15.94:0/4193418251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd0145cf0 0x7f5fd0160da0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 -- 172.21.15.94:0/4193418251 >> 172.21.15.94:0/4193418251 conn(0x7f5fd00b2160 msgr2=0x7f5fd00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:03.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 -- 172.21.15.94:0/4193418251 shutdown_connections 2024-05-27T12:38:03.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.522+0000 7f5fcf200640 1 -- 172.21.15.94:0/4193418251 wait complete. 2024-05-27T12:38:03.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.524+0000 7f5fcf200640 1 Processor -- start 2024-05-27T12:38:03.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.524+0000 7f5fcf200640 1 -- start start 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 0x7f5fd015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 0x7f5fd01f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd006ec80 con 0x7f5fd0145cf0 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd006eb00 con 0x7f5fd01460d0 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fcf200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd0072470 con 0x7f5fd00ce610 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fc5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fc5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39194/0 (socket says 172.21.15.94:39194) 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.525+0000 7f5fc5400640 1 -- 172.21.15.94:0/2960460299 learned_addr learned my addr 172.21.15.94:0/2960460299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fcde00640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 0x7f5fd015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fce800640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 0x7f5fd01f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 -- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 msgr2=0x7f5fd015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 0x7f5fd015ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 -- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 msgr2=0x7f5fd01f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 0x7f5fd01f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 -- 172.21.15.94:0/2960460299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd00aaba0 con 0x7f5fd0145cf0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fcde00640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 0x7f5fd015ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fce800640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 0x7f5fd01f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc5400640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800e870 tx=0x7f5fb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8017070 con 0x7f5fd0145cf0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb800c940 con 0x7f5fd0145cf0 2024-05-27T12:38:03.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd00aaee0 con 0x7f5fd0145cf0 2024-05-27T12:38:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800cd30 con 0x7f5fd0145cf0 2024-05-27T12:38:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.526+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd00ab720 con 0x7f5fd0145cf0 2024-05-27T12:38:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.527+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd0075460 con 0x7f5fd0145cf0 2024-05-27T12:38:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.528+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f5fb801a070 con 0x7f5fd0145cf0 2024-05-27T12:38:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.529+0000 7f5fc6a00640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 0x7f5fac06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.530+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5fb808bef0 con 0x7f5fd0145cf0 2024-05-27T12:38:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.530+0000 7f5fcde00640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 0x7f5fac06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:03.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.531+0000 7f5fcde00640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 0x7f5fac06ef60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0034000 tx=0x7f5fc002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.534+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fd00aaee0 con 0x7f5fd0145cf0 2024-05-27T12:38:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.815+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5fd00cf280 con 0x7f5fd0145cf0 2024-05-27T12:38:03.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.816+0000 7f5fc6a00640 1 -- 172.21.15.94:0/2960460299 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2110 (secure 0 0 0) 0x7f5fb8059f70 con 0x7f5fd0145cf0 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:38:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 204, 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:38:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:38:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2893, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013843436, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24298938368, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748795174912, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "read_bytes_sec": 255, 2024-05-27T12:38:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 5373, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 2 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:38:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:38:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (19s)\n [============================] ", 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:38:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:03.823 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 msgr2=0x7f5fac06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 0x7f5fac06ef60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0034000 tx=0x7f5fc002e000 comp rx=0 tx=0).stop 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 msgr2=0x7f5fd00aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800e870 tx=0x7f5fb800ed30 comp rx=0 tx=0).stop 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 shutdown_connections 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f5fac06caa0 0x7f5fac06ef60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd01460d0 0x7f5fd01f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fd0145cf0 0x7f5fd00aed50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 --2- 172.21.15.94:0/2960460299 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fd00ce610 0x7f5fd015ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:03.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 >> 172.21.15.94:0/2960460299 conn(0x7f5fd00b2160 msgr2=0x7f5fd0071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:03.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 shutdown_connections 2024-05-27T12:38:03.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:03.819+0000 7f5fcf200640 1 -- 172.21.15.94:0/2960460299 wait complete. 2024-05-27T12:38:08.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:38:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.049+0000 7f6cd8a00640 1 Processor -- start 2024-05-27T12:38:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.049+0000 7f6cd8a00640 1 -- start start 2024-05-27T12:38:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd8a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6cd4144bb0 0x7f6cd40cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd8a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cd4072630 0x7f6cd40cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd8a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd40ce200 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd8a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd4144f90 con 0x7f6cd4144bb0 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd8a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd40cdc40 con 0x7f6cd4072630 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd1e00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f6cd4144bb0 0x7f6cd40cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd1400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f6cd40cd6c0 0x7f6cd4071f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:45908/0 (socket says 172.21.15.94:45908) 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.051+0000 7f6cd1400640 1 -- 172.21.15.94:0/856881174 learned_addr learned my addr 172.21.15.94:0/856881174 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd2800640 1 --2- 172.21.15.94:0/856881174 >> v2:172.21.15.97:3300/0 conn(0x7f6cd4072630 0x7f6cd40cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd1e00640 1 -- 172.21.15.94:0/856881174 >> v2:172.21.15.97:3300/0 conn(0x7f6cd4072630 msgr2=0x7f6cd40cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/856881174 >> v2:172.21.15.97:3300/0 conn(0x7f6cd4072630 0x7f6cd40cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd1e00640 1 -- 172.21.15.94:0/856881174 >> v1:172.21.15.94:6789/0 conn(0x7f6cd40cd6c0 legacy=0x7f6cd4071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd0a00640 1 -- 172.21.15.94:0/856881174 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124527343 0 0) 0x7f6cd40ce200 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.052+0000 7f6cd1e00640 1 -- 172.21.15.94:0/856881174 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd40cdf20 con 0x7f6cd4144bb0 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.053+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/856881174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd4144bb0 0x7f6cd40cd180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40cdb20 tx=0x7f6cc402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633d0b406c68ae89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd0a00640 1 -- 172.21.15.94:0/856881174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4037070 con 0x7f6cd4144bb0 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd0a00640 1 -- 172.21.15.94:0/856881174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4032440 con 0x7f6cd4144bb0 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd0a00640 1 -- 172.21.15.94:0/856881174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032810 con 0x7f6cd4144bb0 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 -- 172.21.15.94:0/856881174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd4144bb0 msgr2=0x7f6cd40cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/856881174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd4144bb0 0x7f6cd40cd180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40cdb20 tx=0x7f6cc402a1a0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 -- 172.21.15.94:0/856881174 shutdown_connections 2024-05-27T12:38:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/856881174 >> v2:172.21.15.97:3300/0 conn(0x7f6cd4072630 0x7f6cd40cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/856881174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd4144bb0 0x7f6cd40cd180 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 -- 172.21.15.94:0/856881174 >> 172.21.15.94:0/856881174 conn(0x7f6cd40b3830 msgr2=0x7f6cd40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.054+0000 7f6cd8a00640 1 -- 172.21.15.94:0/856881174 shutdown_connections 2024-05-27T12:38:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.055+0000 7f6cd8a00640 1 -- 172.21.15.94:0/856881174 wait complete. 2024-05-27T12:38:09.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.056+0000 7f6cd8a00640 1 Processor -- start 2024-05-27T12:38:09.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.056+0000 7f6cd8a00640 1 -- start start 2024-05-27T12:38:09.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 0x7f6cd4141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd4144bb0 0x7f6cd41d2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4145330 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd40ceb20 con 0x7f6cd4144bb0 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.057+0000 7f6cd8a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd40cf580 con 0x7f6cd4072630 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39218/0 (socket says 172.21.15.94:39218) 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 -- 172.21.15.94:0/2210826677 learned_addr learned my addr 172.21.15.94:0/2210826677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd2800640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd4144bb0 0x7f6cd41d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 0x7f6cd4141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 -- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 msgr2=0x7f6cd4141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 0x7f6cd4141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 -- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd4144bb0 msgr2=0x7f6cd41d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd4144bb0 0x7f6cd41d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 -- 172.21.15.94:0/2210826677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd41d2a90 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 0x7f6cd4141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.058+0000 7f6cd1400640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c500 tx=0x7f6cc000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.059+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0018070 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.059+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc0007580 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.059+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0004a20 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.059+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd4141fa0 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.059+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd4142560 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.060+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd4075460 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.061+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f6cc0004bc0 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.062+0000 7f6cca000640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 0x7f6cb406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:09.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.062+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6cc008b1b0 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.063+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 0x7f6cb406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:09.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.064+0000 7f6cd1e00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 0x7f6cb406ef60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:09.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.067+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cd41d2a90 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.334+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6cd41ea3d0 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.335+0000 7f6cca000640 1 -- 172.21.15.94:0/2210826677 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7f6cc0059350 con 0x7f6cd40cd6c0 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:38:09.335 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:09.336 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 209, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:38:09.337 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2893, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013866081, 2024-05-27T12:38:09.338 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24298987520, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748795125760, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 2644, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 0 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:38:09.339 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:38:09.340 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:38:09.341 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 8.0 seconds before exiting (25s)\n [============================] ", 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:09.342 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 msgr2=0x7f6cb406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 0x7f6cb406ef60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).stop 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 msgr2=0x7f6cd40b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c500 tx=0x7f6cc000c9c0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 shutdown_connections 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f6cb406caa0 0x7f6cb406ef60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cd4144bb0 0x7f6cd41d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd40cd6c0 0x7f6cd40b1f00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 --2- 172.21.15.94:0/2210826677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cd4072630 0x7f6cd4141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 >> 172.21.15.94:0/2210826677 conn(0x7f6cd40b3830 msgr2=0x7f6cd41e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 shutdown_connections 2024-05-27T12:38:09.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:09.338+0000 7f6cd8a00640 1 -- 172.21.15.94:0/2210826677 wait complete. 2024-05-27T12:38:14.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:38:14.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.570+0000 7fa6d7c00640 1 Processor -- start 2024-05-27T12:38:14.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.571+0000 7fa6d7c00640 1 -- start start 2024-05-27T12:38:14.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6d7c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6d006c940 0x7fa6d006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6d7c00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d006fd30 con 0x7fa6d006ecd0 2024-05-27T12:38:14.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6d7c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d006d260 con 0x7fa6d006f130 2024-05-27T12:38:14.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6d7c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d006f540 con 0x7fa6d006c940 2024-05-27T12:38:14.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6cfe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fa6d006ecd0 0x7fa6d0160990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:43840/0 (socket says 172.21.15.94:43840) 2024-05-27T12:38:14.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.572+0000 7fa6cfe00640 1 -- 172.21.15.94:0/1549061248 learned_addr learned my addr 172.21.15.94:0/1549061248 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263975703 0 0) 0x7fa6d006fd30 con 0x7fa6d006ecd0 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b4003520 con 0x7fa6d006ecd0 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1549061248 >> v2:172.21.15.97:3300/0 conn(0x7fa6d006c940 0x7fa6d006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601317531 0 0) 0x7fa6b4003520 con 0x7fa6d006ecd0 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 >> v2:172.21.15.97:3300/0 conn(0x7fa6d006c940 msgr2=0x7fa6d006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 --2- 172.21.15.94:0/1549061248 >> v2:172.21.15.97:3300/0 conn(0x7fa6d006c940 0x7fa6d006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 >> v1:172.21.15.97:6789/0 conn(0x7fa6d006f130 legacy=0x7fa6d01610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d006f770 con 0x7fa6d006ecd0 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fa6c00019f0 con 0x7fa6d006ecd0 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 >> v1:172.21.15.94:6789/0 conn(0x7fa6d006ecd0 legacy=0x7fa6d0160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 0x7fa6b4003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4004800 0x7fa6b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4005230 0x7fa6b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b400c160 con 0x7fa6b4003ae0 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b400c310 con 0x7fa6b4004800 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.573+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b400c510 con 0x7fa6b4005230 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 0x7fa6b4003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6cfe00640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4004800 0x7fa6b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 -- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4005230 msgr2=0x7fa6b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4005230 0x7fa6b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 -- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4004800 msgr2=0x7fa6b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4004800 0x7fa6b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5200640 1 -- 172.21.15.94:0/1549061248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b4003520 con 0x7fa6b4003ae0 2024-05-27T12:38:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6d5c00640 1 -- 172.21.15.94:0/1549061248 reap_dead start 2024-05-27T12:38:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.574+0000 7fa6cfe00640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4004800 0x7fa6b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:38:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.575+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 0x7fa6b4003ee0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009ac0 tx=0x7fa6b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43187898eacd033 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.575+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8036070 con 0x7fa6b4003ae0 2024-05-27T12:38:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.575+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8031400 con 0x7fa6b4003ae0 2024-05-27T12:38:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.575+0000 7fa6cf400640 1 -- 172.21.15.94:0/1549061248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8009690 con 0x7fa6b4003ae0 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 msgr2=0x7fa6b4003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 0x7fa6b4003ee0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009ac0 tx=0x7fa6b4005830 comp rx=0 tx=0).stop 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1549061248 shutdown_connections 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1549061248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6b4003ae0 0x7fa6b4003ee0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1549061248 >> 172.21.15.94:0/1549061248 conn(0x7fa6d00727d0 msgr2=0x7fa6d0072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1549061248 shutdown_connections 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.576+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1549061248 wait complete. 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.578+0000 7fa6d7c00640 1 Processor -- start 2024-05-27T12:38:14.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.578+0000 7fa6d7c00640 1 -- start start 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4003ae0 0x7fa6d01508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d015b630 0x7fa6d01533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0160470 con 0x7fa6d01f8a20 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0160190 con 0x7fa6d015b630 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6d7c00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d015f0a0 con 0x7fa6b4003ae0 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 -- 172.21.15.94:0/1276533797 learned_addr learned my addr 172.21.15.94:0/1276533797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 -- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4003ae0 msgr2=0x7fa6d01508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4003ae0 0x7fa6d01508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 -- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d015b630 msgr2=0x7fa6d01533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d015b630 0x7fa6d01533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.579+0000 7fa6cfe00640 1 -- 172.21.15.94:0/1276533797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d015bcb0 con 0x7fa6d01f8a20 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6cfe00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0002050 tx=0x7fa6c000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0019070 con 0x7fa6d01f8a20 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c0004b60 con 0x7fa6d01f8a20 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00070d0 con 0x7fa6d01f8a20 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01f7f40 con 0x7fa6d01f8a20 2024-05-27T12:38:14.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.580+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d0158d50 con 0x7fa6d01f8a20 2024-05-27T12:38:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.581+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0075460 con 0x7fa6d01f8a20 2024-05-27T12:38:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.582+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fa6c001c070 con 0x7fa6d01f8a20 2024-05-27T12:38:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.583+0000 7fa6ccc00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 0x7fa6ac06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.583+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c008c0d0 con 0x7fa6d01f8a20 2024-05-27T12:38:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.584+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 0x7fa6ac06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.585+0000 7fa6d5200640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 0x7fa6ac06ef60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8033000 tx=0x7fa6b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:14.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.588+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6c005a160 con 0x7fa6d01f8a20 2024-05-27T12:38:14.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.858+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa6d00ccfa0 con 0x7fa6d01f8a20 2024-05-27T12:38:14.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.859+0000 7fa6ccc00640 1 -- 172.21.15.94:0/1276533797 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7fa6c0059a60 con 0x7fa6d01f8a20 2024-05-27T12:38:14.858 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:14.858 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:38:14.859 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 215, 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:38:14.860 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:38:14.861 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2893, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013866081, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24298987520, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748795125760, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 2644, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 0 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:38:14.862 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:14.863 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:38:14.864 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": { 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "881e3e34-eacc-4585-863c-e2f0c9ad7a0d": { 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (29s)\n [============================] ", 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": 1, 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: "add_to_ceph_s": true 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:14.865 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 msgr2=0x7fa6ac06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 0x7fa6ac06ef60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8033000 tx=0x7fa6b802e000 comp rx=0 tx=0).stop 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 msgr2=0x7fa6d015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0002050 tx=0x7fa6c000b4a0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 shutdown_connections 2024-05-27T12:38:14.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fa6ac06caa0 0x7fa6ac06ef60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d015b630 0x7fa6d01533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d01f8a20 0x7fa6d015b0f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 --2- 172.21.15.94:0/1276533797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6b4003ae0 0x7fa6d01508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.861+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 >> 172.21.15.94:0/1276533797 conn(0x7fa6d00727d0 msgr2=0x7fa6d00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.862+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 shutdown_connections 2024-05-27T12:38:14.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:14.862+0000 7fa6d7c00640 1 -- 172.21.15.94:0/1276533797 wait complete. 2024-05-27T12:38:19.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-27T12:38:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.093+0000 7f703c000640 1 Processor -- start 2024-05-27T12:38:20.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.094+0000 7f703c000640 1 -- start start 2024-05-27T12:38:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.095+0000 7f703c000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7034144bb0 0x7f70340cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.095+0000 7f703c000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f70340cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.095+0000 7f703c000640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70340ce200 con 0x7f70340cd6c0 2024-05-27T12:38:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f703c000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7034144f90 con 0x7f7034144bb0 2024-05-27T12:38:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f703c000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f70340cdc40 con 0x7f7034072630 2024-05-27T12:38:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f7038c00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f70340cd6c0 0x7f7034071f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38102/0 (socket says 172.21.15.94:38102) 2024-05-27T12:38:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f7038c00640 1 -- 172.21.15.94:0/2461344219 learned_addr learned my addr 172.21.15.94:0/2461344219 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:20.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f703a000640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f70340cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f7039600640 1 --2- 172.21.15.94:0/2461344219 >> v2:172.21.15.97:3300/0 conn(0x7f7034144bb0 0x7f70340cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.096+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045498423 0 0) 0x7f70340ce200 con 0x7f70340cd6c0 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f701c003520 con 0x7f70340cd6c0 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2800514971 0 0) 0x7f701c003520 con 0x7f70340cd6c0 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 msgr2=0x7f70340cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f70340cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 >> v2:172.21.15.97:3300/0 conn(0x7f7034144bb0 msgr2=0x7f70340cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 --2- 172.21.15.94:0/2461344219 >> v2:172.21.15.97:3300/0 conn(0x7f7034144bb0 0x7f70340cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340cdf20 con 0x7f70340cd6c0 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f7020001670 con 0x7f70340cd6c0 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 >> v1:172.21.15.94:6789/0 conn(0x7f70340cd6c0 legacy=0x7f7034071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f701c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f701c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c004d40 0x7f701c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7034144f90 con 0x7f701c003a90 2024-05-27T12:38:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70340cdc40 con 0x7f701c004d40 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f701c00ba70 con 0x7f7034072630 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.097+0000 7f7039600640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f701c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7039600640 1 -- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 msgr2=0x7f701c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7038c00640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f701c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7039600640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034072630 0x7f701c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f703a000640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c004d40 0x7f701c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7039600640 1 -- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c004d40 msgr2=0x7f701c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7039600640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c004d40 0x7f701c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.098+0000 7f7039600640 1 -- 172.21.15.94:0/2461344219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c003520 con 0x7f701c003a90 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.099+0000 7f7039600640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f701c003eb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7028009ac0 tx=0x7f701c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72607d557008b8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.099+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028036070 con 0x7f701c003a90 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.099+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028031400 con 0x7f701c003a90 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.099+0000 7f7033e00640 1 -- 172.21.15.94:0/2461344219 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028009690 con 0x7f701c003a90 2024-05-27T12:38:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 -- 172.21.15.94:0/2461344219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 msgr2=0x7f701c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 --2- 172.21.15.94:0/2461344219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f701c003eb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7028009ac0 tx=0x7f701c005830 comp rx=0 tx=0).stop 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703a000640 1 -- 172.21.15.94:0/2461344219 reap_dead start 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 -- 172.21.15.94:0/2461344219 shutdown_connections 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 -- 172.21.15.94:0/2461344219 >> 172.21.15.94:0/2461344219 conn(0x7f70340b3830 msgr2=0x7f70340b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 -- 172.21.15.94:0/2461344219 shutdown_connections 2024-05-27T12:38:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.100+0000 7f703c000640 1 -- 172.21.15.94:0/2461344219 wait complete. 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.102+0000 7f703c000640 1 Processor -- start 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.102+0000 7f703c000640 1 -- start start 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034135c20 0x7f70341599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7034159ef0 0x7f70341386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7034145330 con 0x7f701c003a90 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70340ceb20 con 0x7f7034159ef0 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f703c000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f70340cf040 con 0x7f7034135c20 2024-05-27T12:38:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49940/0 (socket says 172.21.15.94:49940) 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 -- 172.21.15.94:0/2880076584 learned_addr learned my addr 172.21.15.94:0/2880076584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 -- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034135c20 msgr2=0x7f70341599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034135c20 0x7f70341599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 -- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7034159ef0 msgr2=0x7f70341386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7034159ef0 0x7f70341386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.103+0000 7f7039600640 1 -- 172.21.15.94:0/2880076584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703415bf70 con 0x7f701c003a90 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f7039600640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7028034040 tx=0x7f7028004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028040020 con 0x7f701c003a90 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70280038d0 con 0x7f701c003a90 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028036040 con 0x7f701c003a90 2024-05-27T12:38:20.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703415c250 con 0x7f701c003a90 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.104+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703415c810 con 0x7f701c003a90 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.105+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703415a310 con 0x7f701c003a90 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.106+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f7028043020 con 0x7f701c003a90 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.107+0000 7f7031600640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 0x7f701406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.107+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f70280abfc0 con 0x7f701c003a90 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.108+0000 7f7038c00640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 0x7f701406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.109+0000 7f7038c00640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 0x7f701406ef60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7020001200 tx=0x7f7020009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.112+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f702804a070 con 0x7f701c003a90 2024-05-27T12:38:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.392+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f70340cfb40 con 0x7f701c003a90 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.393+0000 7f7031600640 1 -- 172.21.15.94:0/2880076584 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1821 (secure 0 0 0) 0x7f702807a160 con 0x7f701c003a90 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "b6a8d677-9e12-4245-aa4c-e7c6ae09d474", 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stdout: "health": { 2024-05-27T12:38:20.392 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "HEALTH_OK", 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "checks": {}, 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "mutes": [] 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "election_epoch": 4, 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum": [ 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: 0, 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: 1, 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: 2 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_names": [ 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "a", 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "b", 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: "c" 2024-05-27T12:38:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "quorum_age": 220, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": { 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 1, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min_mon_release_name": "squid", 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "num_mons": 3 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap": { 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 33, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "num_osds": 8, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "num_up_osds": 8, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_up_since": 1716813291, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "num_in_osds": 8, 2024-05-27T12:38:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_in_since": 1716813283, 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: "num_remapped_pgs": 0 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap": { 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: "pgs_by_state": [ 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: "state_name": "active+clean", 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: "count": 97 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:20.395 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pgs": 97, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "num_pools": 3, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "num_objects": 2893, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "data_bytes": 12013866081, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 24298987520, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_avail": 748795125760, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_total": 773094113280, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "write_bytes_sec": 0, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "read_op_per_sec": 0, 2024-05-27T12:38:20.396 INFO:teuthology.orchestra.run.smithi094.stdout: "write_op_per_sec": 0 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "fsmap": { 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 18, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 2, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "by_rank": [ 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "filesystem_id": 2, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rank": 0, 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "a", 2024-05-27T12:38:20.397 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "up:active", 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4971 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "up:standby": 3 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "mgrmap": { 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "num_standbys": 1, 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-05-27T12:38:20.398 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "servicemap": { 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 6, 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-05-27T12:37:14.911182+0000", 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {} 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout: "progress_events": {} 2024-05-27T12:38:20.399 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-05-27T12:38:20.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 msgr2=0x7f701406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 0x7f701406ef60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7020001200 tx=0x7f7020009040 comp rx=0 tx=0).stop 2024-05-27T12:38:20.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 msgr2=0x7f703415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7028034040 tx=0x7f7028004730 comp rx=0 tx=0).stop 2024-05-27T12:38:20.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 shutdown_connections 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f701406caa0 0x7f701406ef60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7034159ef0 0x7f70341386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7034135c20 0x7f70341599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 --2- 172.21.15.94:0/2880076584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f701c003a90 0x7f703415b820 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.395+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 >> 172.21.15.94:0/2880076584 conn(0x7f70340b3830 msgr2=0x7f70340cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.396+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 shutdown_connections 2024-05-27T12:38:20.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.396+0000 7f703c000640 1 -- 172.21.15.94:0/2880076584 wait complete. 2024-05-27T12:38:20.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-27T12:38:20.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.622+0000 7f4fd6600640 1 Processor -- start 2024-05-27T12:38:20.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.622+0000 7f4fd6600640 1 -- start start 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fd0161e00 0x7f4fd00cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4fd006fdc0 0x7f4fd00ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd00ce180 0x7f4fd00cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd00ce690 con 0x7f4fd006fdc0 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd006f2e0 con 0x7f4fd0161e00 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fd6600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd006f5c0 con 0x7f4fd00ce180 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.624+0000 7f4fcfe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4fd006fdc0 0x7f4fd00ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4fd006fdc0 0x7f4fd00ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49944/0 (socket says 172.21.15.94:49944) 2024-05-27T12:38:20.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 -- 172.21.15.94:0/2625008830 learned_addr learned my addr 172.21.15.94:0/2625008830 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fd5c00640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd00ce180 0x7f4fd00cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 -- 172.21.15.94:0/2625008830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd00ce180 msgr2=0x7f4fd00cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fd5200640 1 --2- 172.21.15.94:0/2625008830 >> v2:172.21.15.97:3300/0 conn(0x7f4fd0161e00 0x7f4fd00cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd00ce180 0x7f4fd00cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 -- 172.21.15.94:0/2625008830 >> v2:172.21.15.97:3300/0 conn(0x7f4fd0161e00 msgr2=0x7f4fd00cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/2625008830 >> v2:172.21.15.97:3300/0 conn(0x7f4fd0161e00 0x7f4fd00cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.625+0000 7f4fcfe00640 1 -- 172.21.15.94:0/2625008830 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd006f8a0 con 0x7f4fd006fdc0 2024-05-27T12:38:20.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.626+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd006fdc0 0x7f4fd00ca710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00b40e0 tx=0x7f4fb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b025e9475157c70b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.626+0000 7f4fcf400640 1 -- 172.21.15.94:0/2625008830 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8037070 con 0x7f4fd006fdc0 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.626+0000 7f4fcf400640 1 -- 172.21.15.94:0/2625008830 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb8032440 con 0x7f4fd006fdc0 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.626+0000 7f4fcf400640 1 -- 172.21.15.94:0/2625008830 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8032810 con 0x7f4fd006fdc0 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 -- 172.21.15.94:0/2625008830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd006fdc0 msgr2=0x7f4fd00ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd006fdc0 0x7f4fd00ca710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00b40e0 tx=0x7f4fb802a270 comp rx=0 tx=0).stop 2024-05-27T12:38:20.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 -- 172.21.15.94:0/2625008830 shutdown_connections 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd00ce180 0x7f4fd00cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 --2- 172.21.15.94:0/2625008830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd006fdc0 0x7f4fd00ca710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 --2- 172.21.15.94:0/2625008830 >> v2:172.21.15.97:3300/0 conn(0x7f4fd0161e00 0x7f4fd00cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 -- 172.21.15.94:0/2625008830 >> 172.21.15.94:0/2625008830 conn(0x7f4fd00733d0 msgr2=0x7f4fd009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 -- 172.21.15.94:0/2625008830 shutdown_connections 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.627+0000 7f4fd6600640 1 -- 172.21.15.94:0/2625008830 wait complete. 2024-05-27T12:38:20.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.629+0000 7f4fd6600640 1 Processor -- start 2024-05-27T12:38:20.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.629+0000 7f4fd6600640 1 -- start start 2024-05-27T12:38:20.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd00ce180 0x7f4fd015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd0151260 0x7f4fd015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00cb5b0 con 0x7f4fd0161e00 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00cb430 con 0x7f4fd00ce180 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fd6600640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00d1350 con 0x7f4fd0151260 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49954/0 (socket says 172.21.15.94:49954) 2024-05-27T12:38:20.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.630+0000 7f4fcfe00640 1 -- 172.21.15.94:0/1181420942 learned_addr learned my addr 172.21.15.94:0/1181420942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 -- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd0151260 msgr2=0x7f4fd015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd0151260 0x7f4fd015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 -- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd00ce180 msgr2=0x7f4fd015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fd5200640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd00ce180 0x7f4fd015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd00ce180 0x7f4fd015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 -- 172.21.15.94:0/1181420942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd00b6f00 con 0x7f4fd0161e00 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fcfe00640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8037040 con 0x7f4fd0161e00 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb8042070 con 0x7f4fd0161e00 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8030bb0 con 0x7f4fd0161e00 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd0150b20 con 0x7f4fd0161e00 2024-05-27T12:38:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.631+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd01f4b80 con 0x7f4fd0161e00 2024-05-27T12:38:20.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.632+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd0067b70 con 0x7f4fd0161e00 2024-05-27T12:38:20.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.633+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f4fb8047460 con 0x7f4fd0161e00 2024-05-27T12:38:20.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.634+0000 7f4fccc00640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 0x7f4fac06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:20.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.634+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4fb80acad0 con 0x7f4fd0161e00 2024-05-27T12:38:20.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.635+0000 7f4fd5200640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 0x7f4fac06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:20.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.636+0000 7f4fd5200640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 0x7f4fac06ef60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0008ed0 tx=0x7f4fc0008420 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:20.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.639+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fd0067b70 con 0x7f4fd0161e00 2024-05-27T12:38:20.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.866+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fd0150d30 con 0x7f4fac06caa0 2024-05-27T12:38:20.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.881+0000 7f4fccc00640 1 -- 172.21.15.94:0/1181420942 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f4fd0150d30 con 0x7f4fac06caa0 2024-05-27T12:38:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sv1" 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ss1clone1" 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:20.881 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:38:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.883+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 msgr2=0x7f4fac06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.883+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 0x7f4fac06ef60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0008ed0 tx=0x7f4fc0008420 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.883+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 msgr2=0x7f4fd01505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.883+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb80046c0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 shutdown_connections 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f4fac06caa0 0x7f4fac06ef60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fd0151260 0x7f4fd015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd0161e00 0x7f4fd01505e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 --2- 172.21.15.94:0/1181420942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd00ce180 0x7f4fd015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 >> 172.21.15.94:0/1181420942 conn(0x7f4fd00733d0 msgr2=0x7f4fd009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 shutdown_connections 2024-05-27T12:38:20.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:20.884+0000 7f4fd6600640 1 -- 172.21.15.94:0/1181420942 wait complete. 2024-05-27T12:38:20.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-27T12:38:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.090+0000 7fc0dfe00640 1 Processor -- start 2024-05-27T12:38:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.090+0000 7fc0dfe00640 1 -- start start 2024-05-27T12:38:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dfe00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc0e0145df0 0x7fc0e00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 0x7fc0e00721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dfe00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e00cdf50 con 0x7fc0e0160140 2024-05-27T12:38:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dfe00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e00726f0 con 0x7fc0e0145df0 2024-05-27T12:38:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dfe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e0161000 con 0x7fc0e0071d40 2024-05-27T12:38:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dea00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fc0e0145df0 0x7fc0e00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0df400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fc0e0160140 0x7fc0e0160570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38116/0 (socket says 172.21.15.94:38116) 2024-05-27T12:38:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0df400640 1 -- 172.21.15.94:0/920849907 learned_addr learned my addr 172.21.15.94:0/920849907 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0dea00640 1 -- 172.21.15.94:0/920849907 >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 msgr2=0x7fc0e00721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.092+0000 7fc0de000640 1 --2- 172.21.15.94:0/920849907 >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 0x7fc0e00721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.093+0000 7fc0dea00640 1 --2- 172.21.15.94:0/920849907 >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 0x7fc0e00721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.093+0000 7fc0dd600640 1 -- 172.21.15.94:0/920849907 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1552427225 0 0) 0x7fc0d8001490 con 0x7fc0e0160140 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.093+0000 7fc0dea00640 1 -- 172.21.15.94:0/920849907 >> v1:172.21.15.94:6789/0 conn(0x7fc0e0160140 legacy=0x7fc0e0160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.093+0000 7fc0dea00640 1 -- 172.21.15.94:0/920849907 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e0160d20 con 0x7fc0e0145df0 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.093+0000 7fc0de000640 1 --2- 172.21.15.94:0/920849907 >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 0x7fc0e00721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T12:38:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dea00640 1 --2- 172.21.15.94:0/920849907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00cda10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00a9490 tx=0x7fc0d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dacbffc5a30115b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dd600640 1 -- 172.21.15.94:0/920849907 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0037070 con 0x7fc0e0145df0 2024-05-27T12:38:21.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dd600640 1 -- 172.21.15.94:0/920849907 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d0032440 con 0x7fc0e0145df0 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dd600640 1 -- 172.21.15.94:0/920849907 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d002ae60 con 0x7fc0e0145df0 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dfe00640 1 -- 172.21.15.94:0/920849907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 msgr2=0x7fc0e00cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.094+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/920849907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00cda10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00a9490 tx=0x7fc0d002a110 comp rx=0 tx=0).stop 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 -- 172.21.15.94:0/920849907 shutdown_connections 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/920849907 >> v2:172.21.15.97:3300/0 conn(0x7fc0e0071d40 0x7fc0e00721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/920849907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00cda10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 -- 172.21.15.94:0/920849907 >> 172.21.15.94:0/920849907 conn(0x7fc0e006b930 msgr2=0x7fc0e006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 -- 172.21.15.94:0/920849907 shutdown_connections 2024-05-27T12:38:21.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.095+0000 7fc0dfe00640 1 -- 172.21.15.94:0/920849907 wait complete. 2024-05-27T12:38:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.096+0000 7fc0dfe00640 1 Processor -- start 2024-05-27T12:38:21.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.096+0000 7fc0dfe00640 1 -- start start 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0e0071d40 0x7fc0e01f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0160140 0x7fc0e015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00cd4f0 con 0x7fc0e0145df0 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00cd210 con 0x7fc0e0071d40 2024-05-27T12:38:21.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.097+0000 7fc0dfe00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e01461d0 con 0x7fc0e0160140 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49976/0 (socket says 172.21.15.94:49976) 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 -- 172.21.15.94:0/2541960391 learned_addr learned my addr 172.21.15.94:0/2541960391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0dea00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0e0071d40 0x7fc0e01f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0df400640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0160140 0x7fc0e015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 -- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0160140 msgr2=0x7fc0e015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0160140 0x7fc0e015bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 -- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0e0071d40 msgr2=0x7fc0e01f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0e0071d40 0x7fc0e01f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.098+0000 7fc0de000640 1 -- 172.21.15.94:0/2541960391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e01f5110 con 0x7fc0e0145df0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0de000640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c7f0 tx=0x7fc0cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc015070 con 0x7fc0e0145df0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0cc0079d0 con 0x7fc0e0145df0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc007da0 con 0x7fc0e0145df0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e00ccbd0 con 0x7fc0e0145df0 2024-05-27T12:38:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.099+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e00cbe10 con 0x7fc0e0145df0 2024-05-27T12:38:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.100+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e0075460 con 0x7fc0e0145df0 2024-05-27T12:38:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.101+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fc0cc018070 con 0x7fc0e0145df0 2024-05-27T12:38:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.102+0000 7fc0d6a00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 0x7fc0c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.103+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc0cc08bee0 con 0x7fc0e0145df0 2024-05-27T12:38:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.103+0000 7fc0dea00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 0x7fc0c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.104+0000 7fc0dea00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 0x7fc0c006ef60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0034000 tx=0x7fc0d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.107+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0e01f5110 con 0x7fc0e0145df0 2024-05-27T12:38:21.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.341+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0e00cdc50 con 0x7fc0c006caa0 2024-05-27T12:38:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.374+0000 7fc0d6a00640 1 -- 172.21.15.94:0/2541960391 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc0e00cdc50 con 0x7fc0c006caa0 2024-05-27T12:38:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:21.374 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-05-27T12:38:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-05-27T12:38:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ss1" 2024-05-27T12:38:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-05-27T12:38:21.375 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-05-27T12:38:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.376+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 msgr2=0x7fc0c006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.376+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 0x7fc0c006ef60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0034000 tx=0x7fc0d002e000 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 msgr2=0x7fc0e00aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c7f0 tx=0x7fc0cc00cbf0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 shutdown_connections 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fc0c006caa0 0x7fc0c006ef60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0e0160140 0x7fc0e015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0e0145df0 0x7fc0e00aeb60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 --2- 172.21.15.94:0/2541960391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0e0071d40 0x7fc0e01f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 >> 172.21.15.94:0/2541960391 conn(0x7fc0e006b930 msgr2=0x7fc0e015e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 shutdown_connections 2024-05-27T12:38:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.377+0000 7fc0dfe00640 1 -- 172.21.15.94:0/2541960391 wait complete. 2024-05-27T12:38:21.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-27T12:38:21.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.609+0000 7f8fef200640 1 Processor -- start 2024-05-27T12:38:21.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.609+0000 7f8fef200640 1 -- start start 2024-05-27T12:38:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.610+0000 7f8fef200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 0x7f8fe80b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.610+0000 7f8fef200640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe815cfd0 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.610+0000 7f8fef200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe815d2b0 con 0x7f8fe806c040 2024-05-27T12:38:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.610+0000 7f8fef200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe815dbe0 con 0x7f8fe8146060 2024-05-27T12:38:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.611+0000 7f8fe7400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f8fe80cd8a0 0x7f8fe806b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38128/0 (socket says 172.21.15.94:38128) 2024-05-27T12:38:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.611+0000 7f8fe7400640 1 -- 172.21.15.94:0/3767225479 learned_addr learned my addr 172.21.15.94:0/3767225479 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.611+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 0x7f8fe80b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.611+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607949038 0 0) 0x7f8fe815cfd0 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.611+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fcc003520 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659338592 0 0) 0x7f8fcc003520 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 msgr2=0x7f8fe80b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 0x7f8fe80b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 >> v1:172.21.15.97:6789/0 conn(0x7f8fe806c040 legacy=0x7f8fe8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe815d590 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7f8fd4001670 con 0x7f8fe80cd8a0 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 >> v1:172.21.15.94:6789/0 conn(0x7f8fe80cd8a0 legacy=0x7f8fe806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0039e0 0x7f8fcc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 0x7f8fcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc004d40 0x7f8fcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc00ba70 con 0x7f8fcc0039e0 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc00bbf0 con 0x7f8fcc004d40 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc00bdf0 con 0x7f8fe8146060 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0039e0 0x7f8fcc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 -- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 msgr2=0x7f8fcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe8146060 0x7f8fcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 -- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc004d40 msgr2=0x7f8fcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc004d40 0x7f8fcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.612+0000 7f8fe7e00640 1 -- 172.21.15.94:0/3767225479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc003520 con 0x7f8fcc0039e0 2024-05-27T12:38:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.613+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0039e0 0x7f8fcc003e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009a70 tx=0x7f8fcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760aa76f92c4360e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8036070 con 0x7f8fcc0039e0 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd8031400 con 0x7f8fcc0039e0 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fe6a00640 1 -- 172.21.15.94:0/3767225479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8009670 con 0x7f8fcc0039e0 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fef200640 1 -- 172.21.15.94:0/3767225479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0039e0 msgr2=0x7f8fcc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fef200640 1 --2- 172.21.15.94:0/3767225479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fcc0039e0 0x7f8fcc003e00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009a70 tx=0x7f8fcc005830 comp rx=0 tx=0).stop 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fed200640 1 -- 172.21.15.94:0/3767225479 reap_dead start 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fef200640 1 -- 172.21.15.94:0/3767225479 shutdown_connections 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.614+0000 7f8fef200640 1 -- 172.21.15.94:0/3767225479 >> 172.21.15.94:0/3767225479 conn(0x7f8fe80b2240 msgr2=0x7f8fe80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.615+0000 7f8fef200640 1 -- 172.21.15.94:0/3767225479 shutdown_connections 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.615+0000 7f8fef200640 1 -- 172.21.15.94:0/3767225479 wait complete. 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.616+0000 7f8fef200640 1 Processor -- start 2024-05-27T12:38:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.616+0000 7f8fef200640 1 -- start start 2024-05-27T12:38:21.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc0039e0 0x7f8fe80aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe80ac4e0 0x7f8fe80ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80d0110 con 0x7f8fe81e6e40 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8146440 con 0x7f8fcc0039e0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.617+0000 7f8fef200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe80cdd00 con 0x7f8fe80ac4e0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49990/0 (socket says 172.21.15.94:49990) 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 -- 172.21.15.94:0/3905654492 learned_addr learned my addr 172.21.15.94:0/3905654492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fe7400640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe80ac4e0 0x7f8fe80ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc0039e0 0x7f8fe80aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 -- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe80ac4e0 msgr2=0x7f8fe80ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe80ac4e0 0x7f8fe80ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 -- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc0039e0 msgr2=0x7f8fe80aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc0039e0 0x7f8fe80aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 -- 172.21.15.94:0/3905654492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe8143370 con 0x7f8fe81e6e40 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.618+0000 7f8fed200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80b5c30 tx=0x7f8fe000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.619+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0017070 con 0x7f8fe81e6e40 2024-05-27T12:38:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.619+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe0005d60 con 0x7f8fe81e6e40 2024-05-27T12:38:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.619+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000a850 con 0x7f8fe81e6e40 2024-05-27T12:38:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.619+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81e7a20 con 0x7f8fe81e6e40 2024-05-27T12:38:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.619+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81e7fe0 con 0x7f8fe81e6e40 2024-05-27T12:38:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.620+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8068a70 con 0x7f8fe81e6e40 2024-05-27T12:38:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.621+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f8fe000a9f0 con 0x7f8fe81e6e40 2024-05-27T12:38:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.622+0000 7f8fdfe00640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 0x7f8fc8073370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.622+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe008b050 con 0x7f8fe81e6e40 2024-05-27T12:38:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.623+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 0x7f8fc8073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:21.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.624+0000 7f8fe7e00640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 0x7f8fc8073370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8033000 tx=0x7f8fd8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.626+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fe8143370 con 0x7f8fe81e6e40 2024-05-27T12:38:21.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.856+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fe806c690 con 0x7f8fc8070eb0 2024-05-27T12:38:21.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.918+0000 7f8fdfe00640 1 -- 172.21.15.94:0/3905654492 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fe806c690 con 0x7f8fc8070eb0 2024-05-27T12:38:21.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 msgr2=0x7f8fc8073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 0x7f8fc8073370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8033000 tx=0x7f8fd8008030 comp rx=0 tx=0).stop 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 msgr2=0x7f8fe81e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80b5c30 tx=0x7f8fe000cb90 comp rx=0 tx=0).stop 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 shutdown_connections 2024-05-27T12:38:21.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f8fc8070eb0 0x7f8fc8073370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fe81e6e40 0x7f8fe81e72c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe80ac4e0 0x7f8fe80ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 --2- 172.21.15.94:0/3905654492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fcc0039e0 0x7f8fe80aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.921+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 >> 172.21.15.94:0/3905654492 conn(0x7f8fe80b2240 msgr2=0x7f8fe81451b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.922+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 shutdown_connections 2024-05-27T12:38:21.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:21.922+0000 7f8fef200640 1 -- 172.21.15.94:0/3905654492 wait complete. 2024-05-27T12:38:21.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-27T12:38:22.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.135+0000 7fe859800640 1 Processor -- start 2024-05-27T12:38:22.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.135+0000 7fe859800640 1 -- start start 2024-05-27T12:38:22.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe859800640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe85415e930 0x7fe8540b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe859800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe85406fd20 0x7fe8540b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe859800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe85415ed10 con 0x7fe8540adb10 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe859800640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7fe854072930 con 0x7fe85415e930 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe859800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe854072000 con 0x7fe85406fd20 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe852a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe85415e930 0x7fe8540b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe852a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7fe85415e930 0x7fe8540b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50000/0 (socket says 172.21.15.94:50000) 2024-05-27T12:38:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.137+0000 7fe852a00640 1 -- 172.21.15.94:0/1089535512 learned_addr learned my addr 172.21.15.94:0/1089535512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.138+0000 7fe853400640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe85406fd20 0x7fe8540b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.138+0000 7fe852a00640 1 -- 172.21.15.94:0/1089535512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe85406fd20 msgr2=0x7fe8540b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.138+0000 7fe852a00640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe85406fd20 0x7fe8540b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.138+0000 7fe852a00640 1 -- 172.21.15.94:0/1089535512 >> v1:172.21.15.97:6789/0 conn(0x7fe8540adb10 legacy=0x7fe85406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.138+0000 7fe852a00640 1 -- 172.21.15.94:0/1089535512 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8540722e0 con 0x7fe85415e930 2024-05-27T12:38:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.139+0000 7fe852a00640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85415e930 0x7fe8540b4000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe844009870 tx=0x7fe84402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cd376e3eb2a60d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.139+0000 7fe851600640 1 -- 172.21.15.94:0/1089535512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844037070 con 0x7fe85415e930 2024-05-27T12:38:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.139+0000 7fe851600640 1 -- 172.21.15.94:0/1089535512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe844032440 con 0x7fe85415e930 2024-05-27T12:38:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.139+0000 7fe851600640 1 -- 172.21.15.94:0/1089535512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844032810 con 0x7fe85415e930 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 -- 172.21.15.94:0/1089535512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85415e930 msgr2=0x7fe8540b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85415e930 0x7fe8540b4000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe844009870 tx=0x7fe84402a270 comp rx=0 tx=0).stop 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 -- 172.21.15.94:0/1089535512 shutdown_connections 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe85406fd20 0x7fe8540b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 --2- 172.21.15.94:0/1089535512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85415e930 0x7fe8540b4000 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 -- 172.21.15.94:0/1089535512 >> 172.21.15.94:0/1089535512 conn(0x7fe8540b2240 msgr2=0x7fe8540b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 -- 172.21.15.94:0/1089535512 shutdown_connections 2024-05-27T12:38:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.140+0000 7fe859800640 1 -- 172.21.15.94:0/1089535512 wait complete. 2024-05-27T12:38:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.142+0000 7fe859800640 1 Processor -- start 2024-05-27T12:38:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.142+0000 7fe859800640 1 -- start start 2024-05-27T12:38:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe8540adb10 0x7fe8540d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85415e930 0x7fe854157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540b3850 con 0x7fe85406fd20 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540b36d0 con 0x7fe85415e930 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe859800640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fe8540afbf0 con 0x7fe8540adb10 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50008/0 (socket says 172.21.15.94:50008) 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 -- 172.21.15.94:0/689072797 learned_addr learned my addr 172.21.15.94:0/689072797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 -- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe8540adb10 msgr2=0x7fe8540d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe8540adb10 0x7fe8540d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 -- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85415e930 msgr2=0x7fe854157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85415e930 0x7fe854157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.143+0000 7fe852a00640 1 -- 172.21.15.94:0/689072797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe854157ea0 con 0x7fe85406fd20 2024-05-27T12:38:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe852a00640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe844000c00 tx=0x7fe8440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844041020 con 0x7fe85406fd20 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8440306c0 con 0x7fe85406fd20 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe854158180 con 0x7fe85406fd20 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844037040 con 0x7fe85406fd20 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.144+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe854151cb0 con 0x7fe85406fd20 2024-05-27T12:38:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.145+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854068a70 con 0x7fe85406fd20 2024-05-27T12:38:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.146+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7fe844044020 con 0x7fe85406fd20 2024-05-27T12:38:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.148+0000 7fe84aa00640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 0x7fe83406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.148+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe8440ac720 con 0x7fe85406fd20 2024-05-27T12:38:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.148+0000 7fe852000640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 0x7fe83406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.150+0000 7fe852000640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 0x7fe83406ef90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe840000f60 tx=0x7fe8400079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.153+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe84404b030 con 0x7fe85406fd20 2024-05-27T12:38:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.381+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe85406e5d0 con 0x7fe83406cad0 2024-05-27T12:38:22.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.424+0000 7fe84aa00640 1 -- 172.21.15.94:0/689072797 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe85406e5d0 con 0x7fe83406cad0 2024-05-27T12:38:22.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 msgr2=0x7fe83406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 0x7fe83406ef90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe840000f60 tx=0x7fe8400079d0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 msgr2=0x7fe8541524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe844000c00 tx=0x7fe8440046c0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 shutdown_connections 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fe83406cad0 0x7fe83406ef90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85415e930 0x7fe854157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe8540adb10 0x7fe8540d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 --2- 172.21.15.94:0/689072797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe85406fd20 0x7fe8541524b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.426+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 >> 172.21.15.94:0/689072797 conn(0x7fe8540b2240 msgr2=0x7fe8540ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.427+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 shutdown_connections 2024-05-27T12:38:22.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.427+0000 7fe859800640 1 -- 172.21.15.94:0/689072797 wait complete. 2024-05-27T12:38:22.431 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.432+0000 7fe612c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.431 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.432+0000 7fe612c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.431 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.432+0000 7fe612c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.431 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.432+0000 7fe612c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.431 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.432+0000 7fe612c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-27T12:38:22.467 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.468+0000 7fe612200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.467 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.468+0000 7fe612200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.467 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.468+0000 7fe612200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.468 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.468+0000 7fe612200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.468 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-05-27T12:38:22.468+0000 7fe612200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T12:38:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.627+0000 7f1408a00640 1 Processor -- start 2024-05-27T12:38:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.627+0000 7f1408a00640 1 -- start start 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1408a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1404071d40 0x7f14040721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1408a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1404160140 0x7f1404160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1408a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1404160ab0 con 0x7f1404145df0 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1408a00640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f14040726f0 con 0x7f1404071d40 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1408a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1404161000 con 0x7f1404160140 2024-05-27T12:38:22.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1402a00640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f1404071d40 0x7f14040721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1403400640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7f1404145df0 0x7f14040cda10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38136/0 (socket says 172.21.15.94:38136) 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.629+0000 7f1403400640 1 -- 172.21.15.94:0/4221525332 learned_addr learned my addr 172.21.15.94:0/4221525332 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.630+0000 7f1402a00640 1 -- 172.21.15.94:0/4221525332 >> v2:172.21.15.97:3300/0 conn(0x7f1404160140 msgr2=0x7f1404160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.630+0000 7f1402a00640 1 --2- 172.21.15.94:0/4221525332 >> v2:172.21.15.97:3300/0 conn(0x7f1404160140 0x7f1404160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.630+0000 7f1402a00640 1 -- 172.21.15.94:0/4221525332 >> v1:172.21.15.94:6789/0 conn(0x7f1404145df0 legacy=0x7f14040cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.630+0000 7f1402a00640 1 -- 172.21.15.94:0/4221525332 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1404160d20 con 0x7f1404071d40 2024-05-27T12:38:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.630+0000 7f1402000640 1 -- 172.21.15.94:0/4221525332 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401887544 0 0) 0x7f1404160ab0 con 0x7f1404145df0 2024-05-27T12:38:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.631+0000 7f1402a00640 1 --2- 172.21.15.94:0/4221525332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404071d40 0x7f14040721b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14040d0290 tx=0x7f13f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3122e74110a94ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.631+0000 7f1402000640 1 -- 172.21.15.94:0/4221525332 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0037070 con 0x7f1404071d40 2024-05-27T12:38:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.631+0000 7f1402000640 1 -- 172.21.15.94:0/4221525332 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0032440 con 0x7f1404071d40 2024-05-27T12:38:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.631+0000 7f1402000640 1 -- 172.21.15.94:0/4221525332 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f002ae60 con 0x7f1404071d40 2024-05-27T12:38:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 -- 172.21.15.94:0/4221525332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404071d40 msgr2=0x7f14040721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 --2- 172.21.15.94:0/4221525332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404071d40 0x7f14040721b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14040d0290 tx=0x7f13f002a110 comp rx=0 tx=0).stop 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 -- 172.21.15.94:0/4221525332 shutdown_connections 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 --2- 172.21.15.94:0/4221525332 >> v2:172.21.15.97:3300/0 conn(0x7f1404160140 0x7f1404160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 --2- 172.21.15.94:0/4221525332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404071d40 0x7f14040721b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 -- 172.21.15.94:0/4221525332 >> 172.21.15.94:0/4221525332 conn(0x7f140406b930 msgr2=0x7f140406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 -- 172.21.15.94:0/4221525332 shutdown_connections 2024-05-27T12:38:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.632+0000 7f1408a00640 1 -- 172.21.15.94:0/4221525332 wait complete. 2024-05-27T12:38:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.633+0000 7f1408a00640 1 Processor -- start 2024-05-27T12:38:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 -- start start 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1404071d40 0x7f14040ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1404145df0 0x7f14040c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14040cd4f0 con 0x7f1404160140 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14040cd210 con 0x7f1404145df0 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.634+0000 7f1408a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f14041461d0 con 0x7f1404071d40 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50022/0 (socket says 172.21.15.94:50022) 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 -- 172.21.15.94:0/1278852368 learned_addr learned my addr 172.21.15.94:0/1278852368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1402a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1404145df0 0x7f14040c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403400640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1404071d40 0x7f14040ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 -- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1404071d40 msgr2=0x7f14040ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1404071d40 0x7f14040ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 -- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1404145df0 msgr2=0x7f14040c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1404145df0 0x7f14040c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.635+0000 7f1403e00640 1 -- 172.21.15.94:0/1278852368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14040abb40 con 0x7f1404160140 2024-05-27T12:38:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f1403e00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b970 tx=0x7f13fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc019070 con 0x7f1404160140 2024-05-27T12:38:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc00cb60 con 0x7f1404160140 2024-05-27T12:38:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc014440 con 0x7f1404160140 2024-05-27T12:38:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14040abde0 con 0x7f1404160140 2024-05-27T12:38:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.636+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14040ac3a0 con 0x7f1404160140 2024-05-27T12:38:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.637+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1404075460 con 0x7f1404160140 2024-05-27T12:38:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.638+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 91193+0+0 (secure 0 0 0) 0x7f13fc01c070 con 0x7f1404160140 2024-05-27T12:38:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.639+0000 7f13fb400640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 0x7f13e406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.639+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f13fc08bc50 con 0x7f1404160140 2024-05-27T12:38:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.640+0000 7f1403400640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 0x7f13e406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:22.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.641+0000 7f1403400640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 0x7f13e406ef90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.644+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13fc059d60 con 0x7f1404160140 2024-05-27T12:38:22.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.870+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f14040cdd90 con 0x7f13e406cad0 2024-05-27T12:38:22.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.920+0000 7f13fb400640 1 -- 172.21.15.94:0/1278852368 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14040cdd90 con 0x7f13e406cad0 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.922+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 msgr2=0x7f13e406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.922+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 0x7f13e406ef90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000f60 tx=0x7f13f4007b50 comp rx=0 tx=0).stop 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.922+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 msgr2=0x7f14040c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.922+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00b970 tx=0x7f13fc00be30 comp rx=0 tx=0).stop 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 shutdown_connections 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f13e406cad0 0x7f13e406ef90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1404145df0 0x7f14040c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1404160140 0x7f14040c6640 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 --2- 172.21.15.94:0/1278852368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1404071d40 0x7f14040ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:22.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 >> 172.21.15.94:0/1278852368 conn(0x7f140406b930 msgr2=0x7f140415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:22.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 shutdown_connections 2024-05-27T12:38:22.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:22.923+0000 7f1408a00640 1 -- 172.21.15.94:0/1278852368 wait complete. 2024-05-27T12:38:22.943 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-27T12:38:23.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T12:38:23.047 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-05-27T12:38:23.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T12:38:23.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-27T12:38:23.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.250+0000 7f7bef400640 1 Processor -- start 2024-05-27T12:38:23.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.250+0000 7f7bef400640 1 -- start start 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.251+0000 7f7bef400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7be80b4710 0x7f7be81624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.251+0000 7f7bef400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be80adb10 0x7f7be8162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.251+0000 7f7bef400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be815ed10 con 0x7f7be815e930 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.251+0000 7f7bef400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7be8072930 con 0x7f7be80b4710 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.251+0000 7f7bef400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7be8072000 con 0x7f7be80adb10 2024-05-27T12:38:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.252+0000 7f7be7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7be80b4710 0x7f7be81624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.252+0000 7f7be7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7be80b4710 0x7f7be81624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.94:43870/0 (socket says 172.21.15.94:43870) 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.252+0000 7f7be7e00640 1 -- 172.21.15.94:0/3779665580 learned_addr learned my addr 172.21.15.94:0/3779665580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.252+0000 7f7bed400640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be80adb10 0x7f7be8162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.253+0000 7f7be7e00640 1 -- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be80adb10 msgr2=0x7f7be8162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.253+0000 7f7be7e00640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be80adb10 0x7f7be8162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.253+0000 7f7be7e00640 1 -- 172.21.15.94:0/3779665580 >> v1:172.21.15.97:6789/0 conn(0x7f7be815e930 legacy=0x7f7be80b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:23.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.253+0000 7f7be7e00640 1 -- 172.21.15.94:0/3779665580 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be80722e0 con 0x7f7be80b4710 2024-05-27T12:38:23.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.254+0000 7f7be7e00640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81624c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009870 tx=0x7f7bd402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af88ed63cf20f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:23.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.255+0000 7f7be7400640 1 -- 172.21.15.94:0/3779665580 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4037070 con 0x7f7be80b4710 2024-05-27T12:38:23.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.255+0000 7f7be7400640 1 -- 172.21.15.94:0/3779665580 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd4032440 con 0x7f7be80b4710 2024-05-27T12:38:23.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.255+0000 7f7be7400640 1 -- 172.21.15.94:0/3779665580 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4032810 con 0x7f7be80b4710 2024-05-27T12:38:23.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 -- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 msgr2=0x7f7be81624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81624c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009870 tx=0x7f7bd402a270 comp rx=0 tx=0).stop 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 -- 172.21.15.94:0/3779665580 shutdown_connections 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be80adb10 0x7f7be8162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 --2- 172.21.15.94:0/3779665580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81624c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 -- 172.21.15.94:0/3779665580 >> 172.21.15.94:0/3779665580 conn(0x7f7be80b2240 msgr2=0x7f7be80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 -- 172.21.15.94:0/3779665580 shutdown_connections 2024-05-27T12:38:23.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.256+0000 7f7bef400640 1 -- 172.21.15.94:0/3779665580 wait complete. 2024-05-27T12:38:23.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.258+0000 7f7bef400640 1 Processor -- start 2024-05-27T12:38:23.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.258+0000 7f7bef400640 1 -- start start 2024-05-27T12:38:23.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be815e930 0x7f7be81405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7be80b3850 con 0x7f7be80adb10 2024-05-27T12:38:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7be80b36d0 con 0x7f7be80b4710 2024-05-27T12:38:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bef400640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f7be806fd20 con 0x7f7be815e930 2024-05-27T12:38:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50038/0 (socket says 172.21.15.94:50038) 2024-05-27T12:38:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 -- 172.21.15.94:0/2739527675 learned_addr learned my addr 172.21.15.94:0/2739527675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7bed400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be815e930 0x7f7be81405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7be7e00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 -- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be815e930 msgr2=0x7f7be81405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be815e930 0x7f7be81405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 -- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 msgr2=0x7f7be81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.259+0000 7f7beca00640 1 -- 172.21.15.94:0/2739527675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be81541a0 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7beca00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c3c0 tx=0x7f7bdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc0052d0 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bdc005490 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc007260 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be8154480 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.260+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be8154a40 con 0x7f7be80adb10 2024-05-27T12:38:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.261+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be8068a70 con 0x7f7be80adb10 2024-05-27T12:38:23.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.262+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 91295+0+0 (secure 0 0 0) 0x7f7bdc007400 con 0x7f7be80adb10 2024-05-27T12:38:23.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.263+0000 7f7be4c00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 0x7f7bc806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:23.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.263+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7bdc08b5b0 con 0x7f7be80adb10 2024-05-27T12:38:23.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.264+0000 7f7be7e00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 0x7f7bc806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:23.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.265+0000 7f7be7e00640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 0x7f7bc806f080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4034000 tx=0x7f7bd402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:23.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.268+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bdc059570 con 0x7f7be80adb10 2024-05-27T12:38:23.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:23.486+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 --> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f7be806e240 con 0x7f7bc806cbc0 2024-05-27T12:38:23.532 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-05-27T12:38:23.532+0000 7fd94c600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T12:38:23.622 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-05-27T12:38:25.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:25.564+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 9) ==== 91244+0+0 (secure 0 0 0) 0x7f7bdc0554e0 con 0x7f7be80adb10 2024-05-27T12:38:26.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.564+0000 7f7be4c00640 1 -- 172.21.15.94:0/2739527675 <== mgr.4106 v2:172.21.15.97:6832/191062125 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f7be806e240 con 0x7f7bc806cbc0 2024-05-27T12:38:26.564 INFO:teuthology.orchestra.run.smithi094.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-27T12:38:26.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 msgr2=0x7f7bc806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:26.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 0x7f7bc806f080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4034000 tx=0x7f7bd402e000 comp rx=0 tx=0).stop 2024-05-27T12:38:26.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 msgr2=0x7f7be81f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:26.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c3c0 tx=0x7f7bdc00c880 comp rx=0 tx=0).stop 2024-05-27T12:38:26.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 shutdown_connections 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f7bc806cbc0 0x7f7bc806f080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be80b4710 0x7f7be81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7be815e930 0x7f7be81405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 --2- 172.21.15.94:0/2739527675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7be80adb10 0x7f7be81f8880 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 >> 172.21.15.94:0/2739527675 conn(0x7f7be80b2240 msgr2=0x7f7be80b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 shutdown_connections 2024-05-27T12:38:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:26.568+0000 7f7bef400640 1 -- 172.21.15.94:0/2739527675 wait complete. 2024-05-27T12:38:26.589 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-27T12:38:26.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T12:38:26.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T12:38:26.821 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-27T12:38:26.821 INFO:tasks.cephfs.mount:Terminating background process 2024-05-27T12:38:26.821 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:38:26.821 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:38:26.879 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-05-27T12:38:26.879 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T12:38:26.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T12:38:27.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.173+0000 7f844c200640 1 Processor -- start 2024-05-27T12:38:27.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.174+0000 7f844c200640 1 -- start start 2024-05-27T12:38:27.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.176+0000 7f844c200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 0x7f844406c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.176+0000 7f844c200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84440afa00 0x7f844406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.176+0000 7f844c200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84440b5570 con 0x7f8444158a00 2024-05-27T12:38:27.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.176+0000 7f844c200640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f8444158de0 con 0x7f84440afa00 2024-05-27T12:38:27.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.176+0000 7f844c200640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f84440b3dc0 con 0x7f84440af590 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84440afa00 0x7f844406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f84440afa00 0x7f844406ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50070/0 (socket says 172.21.15.94:50070) 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 -- 172.21.15.94:0/2637779514 learned_addr learned my addr 172.21.15.94:0/2637779514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f8448e00640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 0x7f844406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 -- 172.21.15.94:0/2637779514 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 msgr2=0x7f844406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 0x7f844406c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.177+0000 7f844a200640 1 -- 172.21.15.94:0/2637779514 >> v1:172.21.15.97:6789/0 conn(0x7f8444158a00 legacy=0x7f84440b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.178+0000 7f844a200640 1 -- 172.21.15.94:0/2637779514 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84440b40a0 con 0x7f84440afa00 2024-05-27T12:38:27.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.178+0000 7f8448e00640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 0x7f844406c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:38:27.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.179+0000 7f844a200640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84440afa00 0x7f844406ce80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f843c009d40 tx=0x7f843c02a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7cf397e2c8a391 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.180+0000 7f8443e00640 1 -- 172.21.15.94:0/2637779514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c036070 con 0x7f84440afa00 2024-05-27T12:38:27.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.180+0000 7f8443e00640 1 -- 172.21.15.94:0/2637779514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c031600 con 0x7f84440afa00 2024-05-27T12:38:27.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.180+0000 7f8443e00640 1 -- 172.21.15.94:0/2637779514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c0319d0 con 0x7f84440afa00 2024-05-27T12:38:27.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.181+0000 7f844c200640 1 -- 172.21.15.94:0/2637779514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84440afa00 msgr2=0x7f844406ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.181+0000 7f844c200640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84440afa00 0x7f844406ce80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f843c009d40 tx=0x7f843c02a720 comp rx=0 tx=0).stop 2024-05-27T12:38:27.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 -- 172.21.15.94:0/2637779514 shutdown_connections 2024-05-27T12:38:27.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84440afa00 0x7f844406ce80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 --2- 172.21.15.94:0/2637779514 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440af590 0x7f844406c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 -- 172.21.15.94:0/2637779514 >> 172.21.15.94:0/2637779514 conn(0x7f84440727b0 msgr2=0x7f8444072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:27.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 -- 172.21.15.94:0/2637779514 shutdown_connections 2024-05-27T12:38:27.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.182+0000 7f844c200640 1 -- 172.21.15.94:0/2637779514 wait complete. 2024-05-27T12:38:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.186+0000 7f844c200640 1 Processor -- start 2024-05-27T12:38:27.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.186+0000 7f844c200640 1 -- start start 2024-05-27T12:38:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.188+0000 7f844c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84440af590 0x7f84441f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f844c200640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f844c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 0x7f844414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f844c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b4940 con 0x7f8444158a00 2024-05-27T12:38:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f844c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b4660 con 0x7f84440af590 2024-05-27T12:38:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f8448e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.189+0000 7f8449800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84440af590 0x7f84441f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.190+0000 7f8448e00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.94:53380/0 (socket says 172.21.15.94:53380) 2024-05-27T12:38:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.190+0000 7f8448e00640 1 -- 172.21.15.94:0/475341924 learned_addr learned my addr 172.21.15.94:0/475341924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.190+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f84440af0e0 con 0x7f84440afa00 2024-05-27T12:38:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.190+0000 7f844a200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 0x7f844414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f8448e00640 1 -- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84440af590 msgr2=0x7f84441f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f8448e00640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84440af590 0x7f84441f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f8448e00640 1 -- 172.21.15.94:0/475341924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 msgr2=0x7f844414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f8448e00640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 0x7f844414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f8448e00640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441f4580 con 0x7f84440afa00 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.191+0000 7f844a200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 0x7f844414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.192+0000 7f8448e00640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84300109f0 tx=0x7f8430010eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.192+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000ecf0 con 0x7f84440afa00 2024-05-27T12:38:27.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.192+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430004030 con 0x7f84440afa00 2024-05-27T12:38:27.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.192+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844414d6e0 con 0x7f84440afa00 2024-05-27T12:38:27.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.193+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300043e0 con 0x7f84440afa00 2024-05-27T12:38:27.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.193+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f844414dca0 con 0x7f84440afa00 2024-05-27T12:38:27.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.195+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444075460 con 0x7f84440afa00 2024-05-27T12:38:27.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.196+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 9) ==== 91244+0+0 (secure 0 0 0) 0x7f8430003610 con 0x7f84440afa00 2024-05-27T12:38:27.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.197+0000 7f8441600640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 0x7f842406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.197+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f843008e1b0 con 0x7f84440afa00 2024-05-27T12:38:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.197+0000 7f8449800640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 0x7f842406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.198+0000 7f8449800640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 0x7f842406eff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8438007fd0 tx=0x7f8438009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.213+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f843005c570 con 0x7f84440afa00 2024-05-27T12:38:27.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.453+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84440677d0 con 0x7f84440afa00 2024-05-27T12:38:27.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.456+0000 7f8441600640 1 -- 172.21.15.94:0/475341924 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12972 (secure 0 0 0) 0x7f843005be70 con 0x7f84440afa00 2024-05-27T12:38:27.455 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-05-27T12:38:27.456 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"fsid":"b6a8d677-9e12-4245-aa4c-e7c6ae09d474","created":"2024-05-27T12:34:39.612623+0000","modified":"2024-05-27T12:38:26.555261+0000","last_up_change":"2024-05-27T12:34:51.647356+0000","last_in_change":"2024-05-27T12:34:43.107860+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T12:34:52.891189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e188c11-2c56-451c-8851-a9c63593332f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6801","nonce":2544283639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6803","nonce":2544283639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6807","nonce":2544283639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2544283639},{"type":"v1","addr":"172.21.15.94:6805","nonce":2544283639}]},"public_addr":"172.21.15.94:6801/2544283639","cluster_addr":"172.21.15.94:6803/2544283639","heartbeat_back_addr":"172.21.15.94:6807/2544283639","heartbeat_front_addr":"172.21.15.94:6805/2544283639","state":["exists","up"]},{"osd":1,"uuid":"df3b3d8e-1740-4a86-bad2-63ea2e2842e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6809","nonce":1208208615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6811","nonce":1208208615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6815","nonce":1208208615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1208208615},{"type":"v1","addr":"172.21.15.94:6813","nonce":1208208615}]},"public_addr":"172.21.15.94:6809/1208208615","cluster_addr":"172.21.15.94:6811/1208208615","heartbeat_back_addr":"172.21.15.94:6815/1208208615","heartbeat_front_addr":"172.21.15.94:6813/1208208615","state":["exists","up"]},{"osd":2,"uuid":"90511b5d-5c7a-4221-b94f-5d9053d3f15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6819","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6821","nonce":4017177673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6824","nonce":4017177673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6829","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6831","nonce":4017177673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":4017177673},{"type":"v1","addr":"172.21.15.94:6827","nonce":4017177673}]},"public_addr":"172.21.15.94:6821/4017177673","cluster_addr":"172.21.15.94:6824/4017177673","heartbeat_back_addr":"172.21.15.94:6831/4017177673","heartbeat_front_addr":"172.21.15.94:6827/4017177673","state":["exists","up"]},{"osd":3,"uuid":"6d09cbe9-cad3-4dc5-96e9-d8d4b3dc7074","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6817","nonce":1503691431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6820","nonce":1503691431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6830","nonce":1503691431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6823","nonce":1503691431},{"type":"v1","addr":"172.21.15.94:6825","nonce":1503691431}]},"public_addr":"172.21.15.94:6817/1503691431","cluster_addr":"172.21.15.94:6820/1503691431","heartbeat_back_addr":"172.21.15.94:6830/1503691431","heartbeat_front_addr":"172.21.15.94:6825/1503691431","state":["exists","up"]},{"osd":4,"uuid":"5abfea9c-d00e-44f3-8d0c-800ec8cdd3d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6819","nonce":1902436281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6823","nonce":1902436281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6831","nonce":1902436281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":1902436281},{"type":"v1","addr":"172.21.15.97:6828","nonce":1902436281}]},"public_addr":"172.21.15.97:6819/1902436281","cluster_addr":"172.21.15.97:6823/1902436281","heartbeat_back_addr":"172.21.15.97:6831/1902436281","heartbeat_front_addr":"172.21.15.97:6828/1902436281","state":["exists","up"]},{"osd":5,"uuid":"d9c76658-c9fa-4ef3-acf1-45b2e173b8f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6809","nonce":596380992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6811","nonce":596380992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6815","nonce":596380992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":596380992},{"type":"v1","addr":"172.21.15.97:6813","nonce":596380992}]},"public_addr":"172.21.15.97:6809/596380992","cluster_addr":"172.21.15.97:6811/596380992","heartbeat_back_addr":"172.21.15.97:6815/596380992","heartbeat_front_addr":"172.21.15.97:6813/596380992","state":["exists","up"]},{"osd":6,"uuid":"0d0bdebc-3d4e-474d-8abd-9289fe6c8ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6818","nonce":3234217179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6822","nonce":3234217179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6829","nonce":3234217179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3234217179},{"type":"v1","addr":"172.21.15.97:6826","nonce":3234217179}]},"public_addr":"172.21.15.97:6818/3234217179","cluster_addr":"172.21.15.97:6822/3234217179","heartbeat_back_addr":"172.21.15.97:6829/3234217179","heartbeat_front_addr":"172.21.15.97:6826/3234217179","state":["exists","up"]},{"osd":7,"uuid":"bc92dcb4-e3cd-4c79-818c-68ee1160d690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6801","nonce":3824367189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6803","nonce":3824367189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6807","nonce":3824367189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3824367189},{"type":"v1","addr":"172.21.15.97:6805","nonce":3824367189}]},"public_addr":"172.21.15.97:6801/3824367189","cluster_addr":"172.21.15.97:6803/3824367189","heartbeat_back_addr":"172.21.15.97:6807/3824367189","heartbeat_front_addr":"172.21.15.97:6805/3824367189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.434861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.431859+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.459545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.313035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.561110+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.496031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.582980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T12:34:48.554831+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6834/2304976623":"2024-05-28T12:38:23.533273+0000","172.21.15.94:6832/2304976623":"2024-05-28T12:38:23.533273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T12:38:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.458+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 msgr2=0x7f842406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.458+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 0x7f842406eff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8438007fd0 tx=0x7f8438009040 comp rx=0 tx=0).stop 2024-05-27T12:38:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.458+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 msgr2=0x7f844414eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.458+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84300109f0 tx=0x7f8430010eb0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 shutdown_connections 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f842406cb30 0x7f842406eff0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8444158a00 0x7f844414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f84440afa00 0x7f844414eb40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 --2- 172.21.15.94:0/475341924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84440af590 0x7f84441f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 >> 172.21.15.94:0/475341924 conn(0x7f84440727b0 msgr2=0x7f844406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 shutdown_connections 2024-05-27T12:38:27.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.459+0000 7f844c200640 1 -- 172.21.15.94:0/475341924 wait complete. 2024-05-27T12:38:27.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T12:34:52.891189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T12:38:27.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T12:38:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.681+0000 7fada7a00640 1 Processor -- start 2024-05-27T12:38:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.681+0000 7fada7a00640 1 -- start start 2024-05-27T12:38:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.682+0000 7fada7a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fada006fe70 0x7fada00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.682+0000 7fada7a00640 1 -- --> v1:172.21.15.94:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada015bae0 con 0x7fada006c040 2024-05-27T12:38:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.682+0000 7fada7a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada00cf6e0 con 0x7fada015b700 2024-05-27T12:38:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.682+0000 7fada7a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fada00b3540 con 0x7fada006fe70 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9fe00640 1 --1- >> v1:172.21.15.94:6789/0 conn(0x7fada006c040 0x7fada00b4300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.94:6789/0 says I am v1:172.21.15.94:38140/0 (socket says 172.21.15.94:38140) 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9fe00640 1 -- 172.21.15.94:0/633241048 learned_addr learned my addr 172.21.15.94:0/633241048 (peer_addr_for_me v1:172.21.15.94:0/0) 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fada5a00640 1 --2- 172.21.15.94:0/633241048 >> v2:172.21.15.97:3300/0 conn(0x7fada006fe70 0x7fada00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v1:172.21.15.94:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530700473 0 0) 0x7fada015bae0 con 0x7fada006c040 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 --> v1:172.21.15.94:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad88003520 con 0x7fada006c040 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v1:172.21.15.94:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 996537113 0 0) 0x7fad88003520 con 0x7fada006c040 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 >> v2:172.21.15.97:3300/0 conn(0x7fada006fe70 msgr2=0x7fada00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.683+0000 7fad9f400640 1 --2- 172.21.15.94:0/633241048 >> v2:172.21.15.97:3300/0 conn(0x7fada006fe70 0x7fada00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 >> v1:172.21.15.97:6789/0 conn(0x7fada015b700 legacy=0x7fada006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 --> v1:172.21.15.94:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00b3820 con 0x7fada006c040 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v1:172.21.15.94:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2274170213 0 0) 0x7fad8c001670 con 0x7fada006c040 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 >> v1:172.21.15.94:6789/0 conn(0x7fada006c040 legacy=0x7fada00b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 0x7fad88003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad88004800 0x7fad88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad880051c0 0x7fad8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada00b3540 con 0x7fad880039b0 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800c0f0 con 0x7fad880051c0 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fad8800c2f0 con 0x7fad88004800 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 0x7fad88003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 -- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad88004800 msgr2=0x7fad88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5a00640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad880051c0 0x7fad8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad88004800 0x7fad88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 -- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad880051c0 msgr2=0x7fad8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad880051c0 0x7fad8800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5a00640 1 -- 172.21.15.94:0/633241048 reap_dead start 2024-05-27T12:38:27.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.684+0000 7fada5000640 1 -- 172.21.15.94:0/633241048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88003520 con 0x7fad880039b0 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.685+0000 7fada5000640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 0x7fad88003db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad94003db0 tx=0x7fad88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9e56b36a25bc21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94037070 con 0x7fad880039b0 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9402a9f0 con 0x7fad880039b0 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fad9f400640 1 -- 172.21.15.94:0/633241048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94003980 con 0x7fad880039b0 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fada7a00640 1 -- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 msgr2=0x7fad88003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fada7a00640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 0x7fad88003db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad94003db0 tx=0x7fad88005830 comp rx=0 tx=0).stop 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fada7a00640 1 -- 172.21.15.94:0/633241048 shutdown_connections 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fada7a00640 1 --2- 172.21.15.94:0/633241048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad880039b0 0x7fad88003db0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.686+0000 7fada7a00640 1 -- 172.21.15.94:0/633241048 >> 172.21.15.94:0/633241048 conn(0x7fada00b2240 msgr2=0x7fada00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:27.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.687+0000 7fada7a00640 1 -- 172.21.15.94:0/633241048 shutdown_connections 2024-05-27T12:38:27.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.687+0000 7fada7a00640 1 -- 172.21.15.94:0/633241048 wait complete. 2024-05-27T12:38:27.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.688+0000 7fada7a00640 1 Processor -- start 2024-05-27T12:38:27.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.688+0000 7fada7a00640 1 -- start start 2024-05-27T12:38:27.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 0x7fada01530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fada0151930 0x7fada014a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada00cf3a0 con 0x7fada01577d0 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fada00b3de0 con 0x7fada0151930 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.689+0000 7fada7a00640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7fada00ce3b0 con 0x7fad880039b0 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50088/0 (socket says 172.21.15.94:50088) 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 -- 172.21.15.94:0/3269943267 learned_addr learned my addr 172.21.15.94:0/3269943267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5000640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 0x7fada01530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fad9fe00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fada0151930 0x7fada014a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 -- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 msgr2=0x7fada01530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 0x7fada01530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 -- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fada0151930 msgr2=0x7fada014a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fada0151930 0x7fada014a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 -- 172.21.15.94:0/3269943267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada0152090 con 0x7fada01577d0 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5000640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 0x7fada01530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.690+0000 7fada5a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fada00702d0 tx=0x7fad9800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.691+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98016620 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.691+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9801b070 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.691+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98016a20 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.691+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada0152370 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.691+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada0152930 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.692+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada0068a70 con 0x7fada01577d0 2024-05-27T12:38:27.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.693+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 91244+0+0 (secure 0 0 0) 0x7fad98016bc0 con 0x7fada01577d0 2024-05-27T12:38:27.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.694+0000 7fad9cc00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 0x7fad8006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:27.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.694+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fad9808cc20 con 0x7fada01577d0 2024-05-27T12:38:27.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.694+0000 7fada5000640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 0x7fad8006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.696+0000 7fada5000640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 0x7fad8006f020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad94034000 tx=0x7fad9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:27.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:27.698+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad9805ae40 con 0x7fada01577d0 2024-05-27T12:38:28.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.025+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fada00d0f60 con 0x7fada01577d0 2024-05-27T12:38:28.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.026+0000 7fad9cc00640 1 -- 172.21.15.94:0/3269943267 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fad9805a740 con 0x7fada01577d0 2024-05-27T12:38:28.029 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-05-27T12:38:28.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.034+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 msgr2=0x7fad8006f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.034+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 0x7fad8006f020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fad94034000 tx=0x7fad9402e000 comp rx=0 tx=0).stop 2024-05-27T12:38:28.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.034+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 msgr2=0x7fada0157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.034+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fada00702d0 tx=0x7fad9800f520 comp rx=0 tx=0).stop 2024-05-27T12:38:28.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 shutdown_connections 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7fad8006cb60 0x7fad8006f020 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fada0151930 0x7fada014a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01577d0 0x7fada0157c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 --2- 172.21.15.94:0/3269943267 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad880039b0 0x7fada01530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.035+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 >> 172.21.15.94:0/3269943267 conn(0x7fada00b2240 msgr2=0x7fada0071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.036+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 shutdown_connections 2024-05-27T12:38:28.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.036+0000 7fada7a00640 1 -- 172.21.15.94:0/3269943267 wait complete. 2024-05-27T12:38:28.063 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T12:38:28.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.327+0000 7f4145000640 1 Processor -- start 2024-05-27T12:38:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.327+0000 7f4145000640 1 -- start start 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f4145000640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4140162170 0x7f41400d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f4145000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41400d15a0 0x7f41400d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f4145000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41400d2a80 con 0x7f41400b34e0 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f4145000640 1 -- --> v2:172.21.15.94:3300/0 -- mon_getmap magic: 0 -- 0x7f41400d3650 con 0x7f4140162170 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f4145000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f41400d19d0 con 0x7f41400d15a0 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.328+0000 7f413f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4140162170 0x7f41400d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 --2- >> v2:172.21.15.94:3300/0 conn(0x7f4140162170 0x7f41400d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50090/0 (socket says 172.21.15.94:50090) 2024-05-27T12:38:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 -- 172.21.15.94:0/3469209721 learned_addr learned my addr 172.21.15.94:0/3469209721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413fe00640 1 --2- 172.21.15.94:0/3469209721 >> v2:172.21.15.97:3300/0 conn(0x7f41400d15a0 0x7f41400d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 -- 172.21.15.94:0/3469209721 >> v2:172.21.15.97:3300/0 conn(0x7f41400d15a0 msgr2=0x7f41400d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 --2- 172.21.15.94:0/3469209721 >> v2:172.21.15.97:3300/0 conn(0x7f41400d15a0 0x7f41400d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 -- 172.21.15.94:0/3469209721 >> v1:172.21.15.97:6789/0 conn(0x7f41400b34e0 legacy=0x7f41400b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.329+0000 7f413f400640 1 -- 172.21.15.94:0/3469209721 --> v2:172.21.15.94:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41400d2d20 con 0x7f4140162170 2024-05-27T12:38:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.330+0000 7f413f400640 1 --2- 172.21.15.94:0/3469209721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400d1060 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f41400ceff0 tx=0x7f413002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ce1a63cf168f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.330+0000 7f413e000640 1 -- 172.21.15.94:0/3469209721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130037070 con 0x7f4140162170 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.330+0000 7f413e000640 1 -- 172.21.15.94:0/3469209721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4130032440 con 0x7f4140162170 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.330+0000 7f413e000640 1 -- 172.21.15.94:0/3469209721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130032810 con 0x7f4140162170 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 -- 172.21.15.94:0/3469209721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 msgr2=0x7f41400d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 --2- 172.21.15.94:0/3469209721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400d1060 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f41400ceff0 tx=0x7f413002a230 comp rx=0 tx=0).stop 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 -- 172.21.15.94:0/3469209721 shutdown_connections 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 --2- 172.21.15.94:0/3469209721 >> v2:172.21.15.97:3300/0 conn(0x7f41400d15a0 0x7f41400d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 --2- 172.21.15.94:0/3469209721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400d1060 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 -- 172.21.15.94:0/3469209721 >> 172.21.15.94:0/3469209721 conn(0x7f414008bfc0 msgr2=0x7f41400d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 -- 172.21.15.94:0/3469209721 shutdown_connections 2024-05-27T12:38:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.331+0000 7f4145000640 1 -- 172.21.15.94:0/3469209721 wait complete. 2024-05-27T12:38:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.334+0000 7f4145000640 1 Processor -- start 2024-05-27T12:38:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.334+0000 7f4145000640 1 -- start start 2024-05-27T12:38:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41400b34e0 0x7f4140153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41400d15a0 0x7f41400b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41400d07f0 con 0x7f4140162170 2024-05-27T12:38:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41400d32e0 con 0x7f41400d15a0 2024-05-27T12:38:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.335+0000 7f4145000640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 -- 0x7f41400b48d0 con 0x7f41400b34e0 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50102/0 (socket says 172.21.15.94:50102) 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 -- 172.21.15.94:0/3573985089 learned_addr learned my addr 172.21.15.94:0/3573985089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 -- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41400b34e0 msgr2=0x7f4140153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41400b34e0 0x7f4140153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 -- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41400d15a0 msgr2=0x7f41400b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41400d15a0 0x7f41400b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 -- 172.21.15.94:0/3573985089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41400b8dd0 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.336+0000 7f413fe00640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f413800cda0 tx=0x7f413800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.337+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138003b90 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.337+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4138003d30 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.337+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004240 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.337+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41400b75f0 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.337+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41400c8660 con 0x7f4140162170 2024-05-27T12:38:28.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.338+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4140075460 con 0x7f4140162170 2024-05-27T12:38:28.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.340+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 91244+0+0 (secure 0 0 0) 0x7f41380043e0 con 0x7f4140162170 2024-05-27T12:38:28.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.340+0000 7f4137400640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 0x7f412006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T12:38:28.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.341+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f413808c8b0 con 0x7f4140162170 2024-05-27T12:38:28.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.341+0000 7f413f400640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 0x7f412006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T12:38:28.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.343+0000 7f413f400640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 0x7f412006eff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4130034000 tx=0x7f413002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T12:38:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.345+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f413801d340 con 0x7f4140162170 2024-05-27T12:38:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.548+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41400c8860 con 0x7f4140162170 2024-05-27T12:38:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.548+0000 7f4137400640 1 -- 172.21.15.94:0/3573985089 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7f413805ab70 con 0x7f4140162170 2024-05-27T12:38:28.548 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6834/2304976623 2024-05-28T12:38:23.533273+0000 2024-05-27T12:38:28.548 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6832/2304976623 2024-05-28T12:38:23.533273+0000 2024-05-27T12:38:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:listed 2 entries 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 msgr2=0x7f412006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 0x7f412006eff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4130034000 tx=0x7f413002c000 comp rx=0 tx=0).stop 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 msgr2=0x7f41400b9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f413800cda0 tx=0x7f413800a660 comp rx=0 tx=0).stop 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 shutdown_connections 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:6832/191062125,v1:172.21.15.97:6833/191062125] conn(0x7f412006cb30 0x7f412006eff0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41400d15a0 0x7f41400b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4140162170 0x7f41400b9b40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 --2- 172.21.15.94:0/3573985089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41400b34e0 0x7f4140153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T12:38:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.551+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 >> 172.21.15.94:0/3573985089 conn(0x7f414008bfc0 msgr2=0x7f41400b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T12:38:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.552+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 shutdown_connections 2024-05-27T12:38:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-05-27T12:38:28.552+0000 7f4145000640 1 -- 172.21.15.94:0/3573985089 wait complete. 2024-05-27T12:38:28.569 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T12:38:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T12:38:28.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-27T12:38:29.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:38:29.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:38:59.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:38:59.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:39:29.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:39:29.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:39:59.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:39:59.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:40:29.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:40:29.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:41:00.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:41:00.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:41:30.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:41:30.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:42:00.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:42:00.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:42:30.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:42:30.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:43:00.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:43:00.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:43:28.578 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 745, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 1374, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 747, in recv_stderr raise socket.timeout() TimeoutError 2024-05-27T12:43:28.581 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 697, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 1359, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/virtualenv/lib/python3.10/site-packages/paramiko/channel.py", line 699, in recv raise socket.timeout() TimeoutError 2024-05-27T12:43:30.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:43:30.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:44:00.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:44:00.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:44:31.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:44:31.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:45:01.258 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:45:01.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:45:31.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:45:31.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:46:01.539 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:46:01.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:46:31.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:46:31.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:47:01.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:47:01.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:47:31.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:47:31.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:48:02.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:48:02.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:48:32.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:48:32.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:49:02.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:49:02.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:49:32.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:49:32.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:50:02.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:50:02.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:50:32.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:50:32.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:51:02.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:51:02.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:51:32.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:51:32.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:52:03.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:52:03.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:52:33.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:52:33.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:53:03.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:53:03.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:53:33.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:53:33.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:54:03.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:54:03.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:54:33.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:54:33.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:55:03.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:55:03.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:55:34.063 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:55:34.072 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:56:04.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:56:04.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:56:34.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:56:34.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:57:04.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:57:04.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:57:34.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:57:34.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:58:04.753 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:58:04.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:58:34.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:58:34.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:59:05.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:59:05.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:59:35.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:59:35.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:00:05.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:00:05.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:00:35.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:00:35.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:01:05.587 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:01:05.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:01:35.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:01:35.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:02:05.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:02:05.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:02:36.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:02:36.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:03:06.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:03:06.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:03:36.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:03:36.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:04:06.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:04:06.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:04:36.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:04:36.593 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:05:06.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:05:06.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:05:36.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:05:36.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:06:06.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:06:06.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:06:37.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:06:37.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:07:07.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:07:07.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:07:37.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:07:37.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:08:07.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:08:07.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:08:37.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:08:37.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:09:07.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:09:07.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:09:37.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:09:37.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:10:08.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:10:08.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:10:38.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:10:38.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:11:08.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:11:08.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:11:38.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:11:38.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:12:08.691 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:12:08.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:12:38.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:12:38.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:13:08.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:13:08.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:13:39.100 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:13:39.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:14:09.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:14:09.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:14:39.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:14:39.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:15:09.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:15:09.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:15:39.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:15:39.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:16:09.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:16:09.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:16:39.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:16:39.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:17:10.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:17:10.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:17:40.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:17:40.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:18:10.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:18:10.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:18:40.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:18:40.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:19:10.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:19:10.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:19:40.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:19:40.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:20:11.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:20:11.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:20:41.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:20:41.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:21:11.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:21:11.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:21:41.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:21:41.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:22:11.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:22:11.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:22:41.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:22:42.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:23:12.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:23:12.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:23:42.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:23:42.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:24:12.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:24:12.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:24:42.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:24:42.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:25:12.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:25:12.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:25:43.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:25:43.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:26:13.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:26:13.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:26:43.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:26:43.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:27:13.790 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:27:13.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:27:44.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:27:44.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:28:14.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:28:14.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:28:44.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:28:44.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:29:14.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:29:14.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:29:44.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:29:44.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:30:15.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:30:15.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:30:45.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:30:45.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:31:15.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:31:15.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:31:45.525 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:31:45.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:32:15.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:32:15.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:32:45.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:32:45.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:33:16.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:33:16.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:33:46.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:33:46.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:34:16.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:34:16.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:34:46.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:34:46.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:35:16.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:35:16.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:35:47.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:35:47.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:36:17.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:36:17.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:36:47.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:36:47.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:37:17.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:37:17.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:37:47.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:37:47.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:38:18.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:38:18.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:38:48.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:38:48.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:39:18.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:39:18.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:39:48.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:39:48.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:40:18.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:40:18.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:40:49.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:40:49.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:41:19.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:41:19.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:41:49.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:41:49.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:42:19.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:42:19.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:42:49.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:42:49.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:43:20.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:43:20.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:43:50.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:43:50.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:44:20.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:44:20.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:44:50.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:44:50.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:45:20.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:45:20.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:45:51.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:45:51.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:46:21.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:46:21.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:46:51.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:46:51.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:47:21.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:47:21.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:47:51.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:47:51.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:48:22.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:48:22.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:48:52.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:48:52.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:49:22.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:49:22.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:49:52.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:49:52.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:50:22.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:50:22.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:50:52.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:50:52.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:51:23.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:51:23.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:51:53.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:51:53.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:52:23.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:52:23.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:52:53.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:52:53.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:53:24.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:53:24.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:53:54.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:53:54.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:54:24.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:54:24.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:54:54.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:54:54.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:55:24.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:55:24.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:55:54.996 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:55:55.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:56:25.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:56:25.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:56:55.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:56:55.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:57:25.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:57:25.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:57:55.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:57:55.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:58:26.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:58:26.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:58:56.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:58:56.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:59:26.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:59:26.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:59:56.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T13:59:56.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:00:26.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:00:27.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:00:57.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:00:57.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:01:27.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:01:27.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:01:57.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:01:57.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:02:27.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:02:27.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:02:57.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:02:57.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:03:28.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:03:28.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:03:58.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:03:58.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:04:28.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:04:28.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:04:58.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:04:58.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:05:28.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:05:28.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:05:59.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:05:59.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:06:29.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:06:29.368 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:06:59.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:06:59.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:07:29.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:07:29.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:08:00.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:08:00.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:08:30.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:08:30.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:09:00.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:09:00.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:09:30.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:09:30.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:10:00.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:10:00.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:10:30.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:10:30.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:11:01.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:11:01.056 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:11:31.282 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:11:31.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:12:01.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:12:01.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:12:31.703 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:12:31.712 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:13:01.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:13:01.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:13:32.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:13:32.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:14:02.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:14:02.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:14:32.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:14:32.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:15:02.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:15:02.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:15:32.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:15:32.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:16:03.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:16:03.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:16:33.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:16:33.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:17:03.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:17:03.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:17:33.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:17:33.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:18:03.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:18:03.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:18:33.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:18:33.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:19:04.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:19:04.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:19:34.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:19:34.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:20:04.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:20:04.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:20:34.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:20:34.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:21:05.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:21:05.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:21:35.293 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:21:35.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:22:05.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:22:05.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:22:35.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:22:35.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:23:05.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:23:05.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:23:36.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:23:36.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:24:06.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:24:06.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:24:36.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:24:36.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:25:06.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:25:06.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:25:36.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:25:36.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:26:07.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:26:07.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:26:37.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:26:37.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:27:07.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:27:07.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:27:37.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:27:37.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:28:07.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:28:07.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:28:38.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:28:38.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:29:08.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:29:08.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:29:38.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:29:38.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:30:08.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:30:08.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:30:38.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:30:38.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:31:09.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:31:09.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:31:39.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:31:39.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:32:09.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:32:09.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:32:39.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:32:39.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:33:09.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:33:09.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:33:40.130 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:33:40.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:34:10.345 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:34:10.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:34:40.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:34:40.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:35:10.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:35:10.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:35:40.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:35:40.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:36:11.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:36:11.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:36:41.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:36:41.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:37:11.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:37:11.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:37:41.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:37:41.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:38:12.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:38:12.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:38:42.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:38:42.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:39:12.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:39:12.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:39:42.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:39:42.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:40:12.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:40:12.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:40:43.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:40:43.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:41:13.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:41:13.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:41:43.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:41:43.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:42:13.652 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:42:13.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:42:43.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:42:43.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:43:14.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:43:14.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:43:44.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:43:44.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:44:14.535 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:44:14.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:44:44.726 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:44:44.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:45:14.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:45:14.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:45:45.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:45:45.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:46:15.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:46:15.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:46:45.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:46:45.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:47:15.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:47:15.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:47:45.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:47:45.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:48:16.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:48:16.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:48:46.265 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:48:46.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:49:16.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:49:16.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:49:46.652 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:49:46.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:50:16.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:50:16.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:50:46.990 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:50:46.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:51:17.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:51:17.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:51:47.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:51:47.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:52:17.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:52:17.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:52:47.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:52:47.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:53:18.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:53:18.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:53:48.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:53:48.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:54:18.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:54:18.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:54:48.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:54:48.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:55:18.849 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:55:18.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:55:49.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:55:49.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:56:19.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:56:19.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:56:49.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:56:49.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:57:19.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:57:19.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:57:49.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:57:49.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:58:20.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:58:20.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:58:50.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:58:50.377 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T14:58:54.508 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-05-27T14:58:54.509 DEBUG:teuthology.task.console_log:Killing console logger for smithi094 2024-05-27T14:58:54.510 DEBUG:teuthology.task.console_log:Killing console logger for smithi097 2024-05-27T14:58:54.510 DEBUG:teuthology.exit:Finished running handlers